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Development Prograps" and calls for a synthesis of th 
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guidelines in the evaluation of development programs, 
elements in the evaluation of Title programs. (JC) 
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FOREWORD 

Included- here are the prepared papers vhich laid the *'o"ndati°n^°' 
a training Workshop on Evaluating State Title V Programs in the North- 
east. The Workshop vas sponsored by the Northeast Regional Center for 
Rural Devei^opment as a part of its program conducted under Section 503 
(b)(2). Title v' of the Rural Development Act of 1972. 

The Center Was assisted in planning and conducting the Workshop. by 
an ad hoc committee whose members vere: 

■ Edward 0. Moe, Principal Sociologist arid Coordinator, Rural 
Development Programs, Cooperative State Research Service, 
U.S. Department of Agriculture ^ ^ 

Helen Y. Nelson, Professor of Community Service Education, 
College of Human Ecology, Cornell-University 

Howard C. Tankersley,, Program Leader - Rural Development, 
Extension Title V Coordinator, Extension Service, U.S. 
Department of Agriculture 

Joan S. Thomson, Assistant for Cooperative Relations, Cooper- 
ative Extension Service and Title V Project leader. The 
Pennsylvania State University ^ " 

'* The purposes of the Workshop were: 

(1) To clarify the immediate and the longer range purposes 
of evaluating state' Title V pilot programs; 

(2) To develop a conceptual framework for evaluating state 
'fitle V pilot programs in the Northeast ; 

(3) To share evaluation plans among the states of the region 
and to help each state further develop its plans;- 

(U) To explore alternative strategies for conducting evalua- 
tion; . ' , , 
(5) To identify followrup activities and assistance needed 
by the states* in their evaluation work. 

Workshop participants were persons designated by the Title V 
Coordinator in each state in the Northeast as having evaluation, admin- 
istrative, or field .responsibilities for the Title V pilot program. 

Center staff members Lee M. Day, Leslie C. Hyde, and Dorothy J. 
Messenger shared in the details of preparing for and conducting the 
Workshop. Joce3^ Loh typed the manuscript for the Proceedings. 
Photographs are- by LesHe C. Hyde. 
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Olaf F. Larson, Director 
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« 

WHY EVAIUATION OF THE -STATE TITIE V PIIOT EROGRAMS: 
AN .OVERVIEW FROM DIFFE3ffilIT PERSPECTIVES 



Olaf F. Larson* 



One answer to. the question "Why evaluate the statp Title V programs" 
Mght be SK each state is- legally required to ^-'^-f, .^'""al 
der the U.S. Department of Agriculture Reg^ations ?f J^^S^^^^". wril^ 
iegister (October l8, 1973 and in accord with the state Plan Work pre 

i^fei^ provided by a memoranda of ^d-r-*^^i^,^^^y!^"/,?°;,irs;crSlry ' 
institutibn of l862 and the U.S.D.A. and-a^jproved-^by the Assistant^SecreTiary 
S AgScu??u?e. A more academic^- answer might be "Effective evaluation re- 
search irreqaired in order to compare the consequences of a program - the 
TcTvS. goal SSevement - with what was intended and to .seek an e:q)lanatxon 
^ofSscrSa^cies between plan and perfomance". 2 Both answers are valxd 
but neither, I think, would completely satisfy the first purpose of this 
workshop ninely, "to clarify the Wdiate and the longer range purposes of 
eSuatii; Sate TiSe V pilot programs". Before attempting a more complete 
Slower t^the question, it may be usef^a to put Ue more current concerns 
of the Title V evaluation in context . 

1 How/did we get Title V in the Rural Develppnent Act of. 1972? Title 
V caia^th^o^fthrbick door. An ^ Cc^ittee appointed^in N^^^^^ 
as Tresult/^f action, taken by the Division-^of ^gricultur.e. Nation^ Associa 
+Ln of Staie Universities and Land-Grant CoUeges, succeeded in getting what 
,ScLf T^tSSroSid by Sena^^^ Curtis of Nebraska after the key 

develoi^ient bill had been passed by the House. No n°?nSent bms 
So^ component had been included in any of the many ^ 
which had been under consideration by Congress. The ad^hoc Committee e^^ 
countered skepticism in Congress as to the ability of the land-grant xnstitu- 
W^s io df^al development work. A shifting coalition supported what be- 
came Title V. These included spokeanen for" such spoups as the private col- 

Olaf F Larson is Director, Northeast Regional Center for Rural Devel- 
opmentlJd i^^ssor of Rural sociology, Cornell University, Ithaca, New York. 

*The author wishes to acknowledge the helpful comments on an earlier 
dran of the-paper made by Lee M. Day, Director Designate, NERCRD, and by 

a! Wadsworth, Associate Director, Cooperative Extension, Cornell 
University. 

1 The most distinguishing, characteristic of . " ^Jf 
• as the I^asurement oflutcomes. of a project °rf°sram which takes place 

der actual operating conditions or under conditions which reflect in reas 
onlble Sree the problems associated with actual operating progrants . Walter 
SSiLs S Peter H. Rossi and Walter Williams, (eds.) Evaluating Social 
Programs (New York;, Seminar Press, 1972), p. ^' 

2 01 of F- Larson. "Sociological Research Problems" in I«xry R. Whiting 
(ed.), g»:^L^De^omentfpeseich Priorities (Ames: Iowa State University 
Press, 1973), p. 77. ' 
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leges and universities and the f^-yeax colleges and technical schools 
ThI finaa. draft of Title V did not prescribe the "l^f^ 
interest group supporters in a way which would insure their access to funds 
SiS Sght be ai^Lpriated. What, emerged fro. Congress was a pxlot^three- 
year program with authorization of a modest amount of -funds. The Rural De 
veJ^p^ent Act of 1972 was 'signed, somewhat reluctantly, by President Nixon, 
relSantly because it went counter to the. administration' s, proposals for 
rural development. 

1 think it is fair to say that: (l) Congress is waiting for evidence 
that i^he land-grant institutions can do what their spokesmen have ol&xmed, 
(2) enJhusSsm for Title V in the^parts of the Executive Branch crucial for 
supp^rtl^ budget requests has not been great, and (3) some articulate parts 
S^e system of higLr education in the United States are disappointed 
about the lack of role or the. minor role assigned them. 

2 The funds appropriated for Title V have been far below the author- 
ization levels. They, have been meager -in amount and late in coming. The 
amWt for the first year for the Northeastern states ranged f^on> f 
$7,000 for -each of the two components - extension and research - in Rhode 
Island to $U9,000 in Pennsylvania. Thei first year aggregate for the 12 
staSs was about $U30,000 (see Table,, p. 10). Although ^^^1^^ Y^--,"-^f3- 
orized through June 30, 1976, (with a one-year carryover provision for funds 
SpropriSed), the Executive Branch will be making basic decisions jhich 
be£ ?n continuation or r.evision of the program long before that date. 

3. With the meager and' late funds, the Northeastern states have em- 
barked on programs which are diverse in approach, in substantive areas in - 
vSved, and if geographic areas covered. A summary^ statement of what is 
being tried, based on initial plans of work, might be: 

Five states are taking a "grass roots" or "social process" ^PP^o^ch; 
that is the specific substantive extension and research program objectives 
im ;e'«orked out in cooperation with the people ^i^^in the geographic area 
or areas^elected for the pilot effort (Delaware, Massachusetts, New Hamp- 
shire, New York, and Pennsylvania); 

■. TWO states will work in the land-use and l^<i-^^%°°f '2^^,^^^.^'^^°^^^ 
with quite different program emphasis (Hew Jersey and Rhode Island), 

Two states will work on aspects of economic development but ^^th quite 
different emphasis (Maryland and Vermont) ; the Vermont plan includes aspects 
of human resoiirce development; 

Two states (Connecticut and West Virginia)^ are working in the area of 
cormZly services ^ personal health services in ^^^^^^ 
rural fire protection, rural public transportation and community emergency 
medical services in the case of West Virginia; 

One state (Maine) is emphasizing the improvement of rural housing. 

rpoffrauhic areas involved include a single community, two or more com- 
' rnunitSr^siSgS couS^! two or more counties or a multi-county .area, and 
an entire state. (8tee Appendix C) . 
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With this contek, let us return to the, question Why evaluate 
"state Title V pilot programs?" I suggest that the answer, which bears 
on evaluatio^i strategies and on evaluation des^n, may be ^ 
from the perspective of the several audiences who are the potential users 
of the evaluation findings; Bach of these audiences has some specific 
questions irtiich they wduld like to have answered. The 
L the -same from audience to audience. The relative Importance -which a 
given audience attaches to a specifdc question may vary "^^r time. ^ For . 
those who must make decisions about evaluation strategies designs, 
the several audiences may not carry equal weights or the weighting may 
shift over time. 
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The Audiences 

i have identified seven major audiences for Vour consideration: 

1. Congress including its sub-systems, e.g., the Senate Ccannittee 
on Agriculture and Forestry and. its Sub-comittee on Rural Development, 
the Counterpart committee in the House, the Appropriations3,oommittees, ^ 
committee staff members', individual members of the Senate and the 
House and their staff, and, as aja agent for Congress, the General Account- 
ing Office. 

2. The Secretary of Agriculture, (and his surrogates) who Is directed 
and authorized to do certain things under Title V and also under Title VI 
S thrS De veloiinent Act. His surrogates for Title V will include the 
Assistant Secretary for Conservation, Research and Education; the^Coop- 
erative State Research Service; the .Extension ServicV; and the^plm^ning 
Sd evaluation unit attached to the Secretary' s office. Surrogates for. 
S^e SeSeW for his Title VI coordination and goal-fomulating respon- 
sibilUieslIclude the Assistant Sebretary for Rural I?eveloi«nent and tne 
Rural Development Service. 

3. The.Office of the President, including the Office of Management 
and Budget. 

U The National Association of State Universities and iktid-Gfaiit ^ 
Colleg;s Zl its component sub-systems, e.g. , the Di^sion^of A^^^^^^^ 
its council of Administrative Heads of Agriculture, and the 
Rural Development Committee; the Extension Service section, the Extension 
Sittee on^rganization and Policy and ^^^^f °P^!f ^r-f^J^^^i^^.^^Jr ■ 
lative Committee; the Experiment Station section, ^h^/'^^J^^"!?^!^^"" 
Committee on Organization and Policy and its legislative sub -committee , 
and the Council^ of President's . 

5. College administrators and extension and research staff wJ.th 
direct and limnediate .responsibiLity for conducting the state Title V pro- 
grl^s! ?Sgether with the State Rural Development Advisory Council members. 

6 A diverse set' of "publics" and interest groups outside the laijd- 
grant colleges of l862 and the USDA. These renge from the local to- the 

3 In response to a request from Senator Hubert Humphrey, ^Je Coun,^ 
for Agricultural Sciences and Technology, headquartered at Iowa State 
University, has established a Task Force to, r^Jjif^'^j;;.^^!' 
develojment. See News From Cast , vol. 1, no. k (October, 197'^). 



national levels. They may be public or private in nature, organized or 
unorganized. They include the communities and agencies involved in some 
w with the Title V pilot progrmns. They incluae local government off i- 
cilll some in this audience category axe critical of the I'eadershxp role 
assigned the land-grant institutions of 1862. 

7. Scholars and\rofessionals concerned with 'rural development pol- 
icies, programs, and strategies. , • , 

Some Questions From the Audienc es 

Now let us look at some examples of questions to which I can, imagine 
these audiences might like answers, JLeaving aside conventional fiscal 
accounting. 

■ \. congress - There are at least four ma;jor questions to which I assume 
Congress will want answers. They are: 

A Have the state Title V pilot programs been in compliance with the 
* law and been conducted in accord with the intent of Congress? 
For example, specifically: 

\ (1) Has the content of the Extension and the Research programs 

\ been consistent with what the law says they shall be, or majr 

be (Section 502 a and b; this .assumes that no funds will be^ 
appropriated for the small farm programs )? 

(2y To what extent have the audiences 'identified by the law been 
actually served (Section 501a and Section 502 a and bj? 

(3) To what extent were other private and publicly supported col- 
leges and universities, including any College of 1890 in the 
state, a School of Engineering, community colleges, and area 
technical institutes participants in thj^ program? ^ Was the 
program among these educational- institutions co-ordinated? 

(h) To what extent were the discipline resources of the entire 
campus - beyond those typically in colleges of agriculture, 
engineering, and home economics - utilized?"^ 



(5) Did the State Rural Development Advisory Councils have the 
size and composition specified by law and perform the func- 
tions expected (Section 50^e)? 

(6) Was the.pi-ogram co-ordinated with the rural development pro- 
grams of Federal, State,, and local government (Section 505a}? 

(7) What was learned in the extension and research programs about 
techniques and organization structure which could be used in 



5 Ibid , p. l6. 
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U See statement. Explanation of the Rural Development Act of 1972, 
15, made to .he Senate on August 17, 1972 by Senator Heman E. Tai- 
Chairman of the Senate Committee on Agriculture and Forestry in 
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an extended'"and expanded Title V program? 

B. Wh^T^ere the results and impacts of the Title V program in rela- 
tion to the two levels of. purposes or objectives specified in 
flfitle V? • /, . 

(1) At the most general level, what did the program contribute to 
achieving the overall purpose of Title V, i.e., "to encourage 
and foster a balanced national development that provides op- 
portunities for increased numbers of .Americans to work and_ en- 
joy a high quality of life 'dispersed throughout our nation 
(Section 501)? 

^ (2a) More specifically, to what extent were those involved with 

public services and investments in rural areas or who provide 
or may provide employment in rural areas supplied with scien- 
tific, technical, ecohOTfic, organizational,^ environmental, and 
management information knowledge useful to them and to what 
extent were they assisted and encouraged in the interpretation • 
and apDlication of this information to the practical problems . 
and needs in rural development as defined 7 (Sections 501a 
and 502a)? What was the impact of the use of this Information 
in achieving rural development- goals? j. 

(2b) To what extent was knowledge and information developed through 
research and investigation useful to thoke plwining, carrying 
out, managing, or investing in faciUtieb, ser --es, busines- 
ses, or other enterprises - public and pHyate that^may con- 
tribute to rural development as defined (Sections ^Olb and 
502b)? What was the impact of the use <if the j-cssarch results 
- in achieving rural development goals? ■ ■ % 

!• 

C. To what extent have the fuhds provided the Title V program and the 
experience with it enhanced the capabilities of colleges and uni- 
versities to perform the public service roles .of research, trans- 

' fer , and practical application of knowledge in support of rural 
devKqpment as defined (Section 501c)? To what extent has this 
e|^6p£ied the l862 institutions and- their cooperators to allocate 
:^^sources to rural development? 

A "hidden" or Catent question is: "Are the assumptions underlying 
the Title V program valid, i.e., that the institutions of higher 



D. 



6 Ibid, p. 16. 
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"7 "Rural development" means the planning, financing, and development 
of facilities and services in rural areas that contribute to making these 
areas desirable places in which to live and make private and business in- 
^sSents; ?he planning, developnent, and expansion of business and industry 
In ru^al areas to provide increased employment and income; the planning, de- 
ielo^^tt conservation, and use of land', water, and other natural resources 
ofrZal ^reas to maintain or enhance the quality of the environment for 
people and business in rural areas; and processes and procedures that bave^ 
said objectives as their major i>urposes (Section 507aJ . 
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education, under the leadership of the land-grant colleges ^pf 
1862 are capable of making an effective contribution through 
research and extension to the solution of practical problems 
and needs of rural development?" 
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If the answer is affirmative, what is needed to further 
enhance this capability? / 

If the assumptions-are^nbt ^aHd, what are the alternatives 
for providing the scientific inquiry and the educational effort 
to back up rural deve^pment? 

* 

2 The Secretary of A griculture - As the responsible administrator for 
Title V, the Secretary needs the types of information listed above, under 
the first major. question for Congress, since it is- he who must answer 
questions from Congress as 'to whether the program has been conducted in 
conformity with the law and the intent of Congress . 

\ " " 

He also needs information about results and impacts to assist him in 
• "making budget and policy recoransndations to the Office of the ^sident and 
to Congress about the program. To assist in making these recotoendations 
U S seem that the SeSetary a^d his surrogates Jhould have^informtion 
' as to inputs into the pilot extension and research pr^ogrMis by the 1B6^ 
■ institutions, by cooperating institutions in higher education Jy^^^^jl 
agencies, by State agencies, by local goverranent, by .\he local *J««».^ity, . 
^d bv lihe private sector. He should have information to give him. an. aware- 
Sss and ^lerstanding of the processes initiated He^ should ^^av.e the^^ost 
reliable information possible on changes , direct and secondary , positive and 
negative, plaiuied and unplanned, whicH can reasonably b^e attributed to the 
prS^! Considering the short time ^pan allowed for the pilot. effort, xt • 
Ly be necessary to depend heavily on lead indicators of change and on 
evidence, df intermediate outputs. > \ 

•To assist him in making policy recommendations, ^I would also expect 
the Secretary to weldone evidence on the .third and fourth\ major questions _ 
ab^e JdentSied for bongress . To these, ^ he could well add a fifth question, 
i e what is the effectiveness of the T.Hle V program' considering costs and 
benefits. We would hope that benefits would not be limited to those to which 
a dollar sign can be attached. 

Finally, the Secretary necessarily has 'an additional ipterest^ I assume, 
in evaluation of the State Title V pilof programs because of the USDA pre- 
scribed Regulations for their conduct. These regul^itions (Federal Re^^ister 
October 18^ 1973) represenfthe department's interpretation, for purposes .of 
a^nistrative implementation,, of the law and of the intent of Congress 
Questions which a reading of the Regulations suggests should f J^^^ 

evaluation. efforts, in addition to or supplementing questions already iden- 
tified, axe as follows: ^ ^ 

A. What progress was made toward achieving the extension and reseerch 
objectives stated in the approved Plan of Work? 



B. How integrated, how complementary were the extension ana research 
components of the state program? , • ^ \ 



\ 
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cl ,9oK effective were the extension and the research program techniques? 

D. How effective vas the organizational structure for planning -and- 
' conducting the extension and research programs; 

i Did the urogram- give priority to education and research aasistance 
lesAZ Sl^Selsing dob. and incouie opportunities proving quality 
of life, improving essential community services and facil/tiesr^- 
proviS housing aSd hone Improvements, and enhancing those social , 
processes necessary to achieve these goals? 

F. Did the program concentrate on limited geographic or problem areas 
where .TitleV efforts could be expected to have high Impact with;Ln 
the three-year period authorized for the pilot effort? 

3. 0fflP.e of the President As a basis for decision-making on pro^a.8 
i,d budgets to recommend to Cbngress and to support or "PP^^X^^^!^,^^ 
^sumed that the 'Office of tHe.President would have a Jf=i«l^i^*f 
in informaUon on results and -impacts and on program effectiveness, 



k, 'The national A«so >-^t^on oV State Universities and Land-Granf Coileges 
p;rsSctives as to questions to be. answered, b, f J^^f^^ 
dlffSr Mion« -the several key deci«ion^ing points within NASUIGC. 
Hiw^verf Som qaestions already listed fpr other audiences ye 

might expect a special interest in infonnation on: ^ 

A. Uroacts and results, including the achievewent of objectives set 
forth in the Plans of Work. 

•R The #^nLmcement of the capabilities of the institutions; of higher 
^* Wcat5S S Serf oxm pablic service roles in support of rural develop- 
ment, and, • 

c The validity of the assumptions concerning ^the land-grant - institu- 
tioni of lM2- and other destitutions of higher education which were 
made in instituting the Title V rural developnent pilot effort. 

If the assumptions are found- xmwarranted, then there sJjJLd.bj 
interest in knowing why this is^so and what changes the institutions 
^Sd neeftf ^iHrganizlttioh, staffing patterns "^^^PJ^f 
Scentive And reward systems, and- in other yespects to validate the 
assumptions. ^ 

'5. p»ri>fr> Arfa.ini«trators and Others m ^->^ THr,.ct Res ponsibility for" the 
State Title V Programs - Jiven though there were no formal or legal 
recrements fSTlPg so, the administrators, extension and resfearch 
stSs/Sd Ad^iso^ylouniil members responsible for the «tate progr^s 
wJSd have a vested interest in the best information they co^d obtain 
Soi the monitoring and evaluation of the pilot prograss. If I correctly 
interpret their perspecti^ve , their interests would include: 

A. Assessing progress toward stated program objectives; 



B. Assessing extension and' research program techniques, the^rogram 

/, .0012/^ ■ ' 
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organizational structure and related factors as a means of under- 
' standing "why" desired changes occurred or did not occur. In other 
words, they. would be seeking information which could be fed back to 
increase program performance in the specific situationjS in which the 
pilot programs are being condacted; 

C. To acquire knowledge concerning^ strategies, processes, techniques, or* 
ganization; e^Jc, which would be- useful in extending the pilot pro- 
^gram to other areas. 

6. Publics and Interest Groups' Outside of the Land-Qrant 'Colleges of 1862 
and the USDA > Infoimtion on impacts and resultjs, and on prcgram effec- 
tiveness, including! costs and benefits .arid their e-^'^' ' ^^^ion, is ass\Med 
to a common concern, at least potentially, of L '.se publics and 
groups . 

Some, however', haye special concerns. For example, the I890 land- 
grant injsiitutions and other types ofjpublicly and privately supported 
colleges arid universities may well be expected to^ desire evidence that *the 
Title V program was conducted in compliance with the lavr, the intent of 
Congress, and the USDA Regulations with respect to their Involvement in 
the pilot programs. * ' / 

Further, constituencies for some of the audiences for research and 
extension assistance which were identified in th^ legislation will be look- 
ing for evidence that. these audiences were served. Also, advocates of cer- 
tain types of review, approval and c >c.rdination procedm'es which were elim- 
inated from the final draft of W law or from the original drafts of the 
Regulations will be alert for evidence whiph will bear on the position -they 
took. ^ " 

7. Scholars and Professionals - Numerous scholars and professionals, most 
of. whom- are not involved in the iiranediate Title V program operations,. Kill 
have an interest not only in "descriptive" evalua,tion but in "e3q)lanatory 
evaluation. Why or how did bhe pro-am succeed in producing the effects 
that it did? ' . , 

This group is interested in cumulative, generalizable knowledge which 
lends itself to application in other rural development and related situa- 
tions. Some in this group have an interest in using the evaluation re- 
' suits as an aid in policy analysis and in devising, alternative progr^ 
models.. ■ 

In the Northeast what knowledge can^e aggregate from the experiences 
of thea2 states which will be useful fp^generating policy, for improving 
-progi^TprocedureSi and for staff training? Since diverse approaches are 
being used among the 12 states what might be learned from a compaxat?"« 
analysis of the different approaches? 

Summary * .j 

I 

The' extent to which the several audiences I have named ha^^e, 
common or unique questions which evaluation might attempt to answer 
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could be shQwn by a matrix, with columns for the audiences and rows 
for the quesl^ions. • " >. " • 

I an not suggesting that ^alJro?''th« iudierices and the illustrative 
questions should be veighted eqpially fof^the iaaoediate purposes of 
evaluating the state TiUe V programs an4 considering the constraints 
posed by liaijied resources' to do the evaiuat^ion. Meither an I propos- 
■ *s - +httt every state should necessarily attempt to provide answers to 
" . ' tiye set of question; posed; perhaps there is an opportunity for 
' .i division of labor and coopcrative^effort in conducting the evalu- 
ation." The major purposes of the_ monitoring and evaluation might be 
condensed somewhat as follows: 

1. To determine Ihe reWts and Impacts Of the Title V pilot 
extension and research programs and their, effe'ctiveness; 

2. To determine the extent of compliance with the law, the in- 
tent of Congress and the USDA Regulations in conducting the pilot 
programs; ' , / 

3. To determine the extent to which participation in the program 
has enhanced the capabilities of institutions of higher learning to per 
foim' public service role*' in support of rural development; 

1+. To provide some test of the validity of the assumptions con- 
cerning the land-grant institutions of l862.and other colleges and 
universities which underly the establishment of the Title V pilot 
effort; ■ . ' . . 

5. Throu^ i, 2, 3 and k to provide a more informed basis for 
decisions by policy makers and the public a:s to the continuation and 
expansion, modification-, or termination of a Title V type program. 

6. To improve program performance of the Title V pilot efforts^ 
in specific situations; and " , i 

7. To increase generalizable knowledge about rural deyeloiment 
and about the application "of research and extension by institutions 
of higher learning to achieve rural develoipoient objectives in a 
variety of situations. ^ 

The diversity of audiences and the range of their infonM,tion 
needs and interests, cottbined with the constraints and the context 
of ^the- present pilot effort, complicates the decisions i^ich will 
need to be made with respect to evaluation efforts. Some resolution 
of the conceptual, strate£or, design and measurement problems of 
evaluating the Title V programs is what wo will b^e making a start- 
on working through together during the rest of this wor):shop. 
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Northeastern States' Shares of Appropriations for. Title V, 
Rural Developoent Act of 1972, Fiscal Year im 



Ik? l/Cb V W 


State Total 


Extension 

/■ 


Research. 


Connecticut 




$12,920 


$12,920 


DelaTrare 


.15,816 


■7,908 ■ 


" 7,908 


Maine 


23,170 


11,585 


11,585 ■ 


Maryland 




17,723^ 


. 17,723 ■ 


Massachusetts 


29,7^^2 


1U,871 


lU,871 


Hev Hampshire 


18,596 


9,298 


9,298 


Hev Jersey 


29,796 


1U,898 


1U,898 


Hew York 


81,07'* 


U0,537 


\\t\ COT 


Pennsylvania 


98,192 


U9,096 


1*9,096 


Rhode Island 


llv,238 


7,119 


7,119 


Vermont 


20,228 


10,111* 


10,1J.U 


West Virginia 


37.53Q_ 


18;765 


18,765 


Total for 
' 12 states 


^29,668 . 


.$2lUt53J^ 


$2ll*,83U 



Source : GSRS - Op - 1083 - B ,. October 26 , 1973 . 
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EVAIUAriON OF RURAL DEVEDOPMEOT PROGRAMS: 
TOWARD A PARADIGM TO GUIDE THE IMPIEMENTATIOg AND 
EVALUATION OF RURAL DEVELOlMENT PROGRAMS 



J. Patrick Madden 



At least two central beliefs xmderly society's, interest in lyal 
develoment. First is the distributive justice argument-that all peo- 
S! r^Sdiess of their place of residence, deserve equal opportunity 
£ develSl^ their full potential. The second belief is that an alter- 
native toCc over-crowderrnegalopolis can and yst be found An o^de^^^ 
.-rational.,..and..r.apid.develoi«ient of rural axeas f ji^^^J.J^^jHr'nd 
alternative.. The 1970 Census data have revealed a startling new ^rena 
S; mSI ISil areas-r the Wation is increasing rather f f ' 

Ihat^^ done to ensune that the new population growth leads not to a 
feJeriSaSon L qualitj\f life in rural co««anities^t^^ rather, to ar. 
enhancement in the social W economic conditions m rural areas. 

■ Those of us who axe rural" development researchers share the belief _ 

N either cause>and-effSt.or probabilistic)- that will facilitate the dis- 

^ J. Patrick Madden is Professor of Agricultural Econcaaics, The 
^Pennsylvania State University, University Park. 

* A revision of the paper presented at the Workshop. 

^ Research is needed to provide technologies that will enable small 
towns to continue to exist- and provide a decent .place for people to live. 

satisfactory and economical. , 

Research is also needed for the development of new technology to be 
used by Ta^l to ^derate size rural firms, if.^f J^™?' ,f " 
Sere is to develop "intermediate" technology (Schumacher, 1973) that 
Slu enabL the t?pe of firm usually found in rural axeas to compete 
Sth large firms imo have had the benefit of many years of private and . 
publicaily sponsored research. 
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cdvery;' and impleifientation 'of more feasible- and efficient ways to over- 
come these barriers to rural developnent . 

Several treatises have set forth their own versions of the^agenda 
for rixral development research. - What •seems to be needed is a frame- 
^^rklSSin whicfone could estimate (l) the probability that a given 
•piece of reseafch will succeed in making a significant social contri- 
butidn, and (2) the expected social value of this contribution, in 
view of the uncertainties involved. • 

Also needed is a general pulling' together and synthesis of the- 
relevant theoretical and>pirical contributions ^"^^^"^ly .^^^J^Jf J^' 
T^e ^review article by Jansma et al.'is a ^i6^Ji°SJ^^°°"^^^?J^S 
the domain of economics. Likewise, the paper by ^1974) 
pulls together and interprets a massive, array of l^^^f ' 
Soky, social psychology, and other social sciences, including an^ex- 
tenSve discussion of the problems and prospects of measuring social 
well-being changes associated with rural develoiment intervention pro- 
^^s! sfsmon^has produced excellent interdisciplinary research, both 
from the theoretical and empirical points of view (1973a, 1973b J. 
Warren has written a very .-significant theoretical treatise relevant to 
rui-al development, "Alternative Community Paradigms- (197^+ J. 

An Wessive literature is now available; but much^more effort is 
needed in improving both the predictive ability of our ^^^^f ^^J?. 5"^^^ 
oS abilityto apply our knowledge effectively in cooperation with prac- 
tSners-'tLse ifcLrge of making policy decisions and carrying out 
rural development' programs • ^ 

, -HOW can we maximize^ o^^T professional' contr^ibution- to this cause?. ^ 
What can we do to enhance the probability that, at the end of 3 or 5 
6r 25 years of research and experimentation, we will know more than 

now^'k^ a^out the- essential ingredients of a f „^,^^f 
oment effort? Giveii that the current efforts under Title V ol the 
S Development Act of 1^2 are, in effect a jpilot f ^ 
•bTsure the -results will add up to a cumulative body of knowledge, a 
more mature paradigm of rural development than we now possess? 

I The central thesis of this essay is that'^if we are to optimally 
T,..ort„itive we must be fully cognizant of the vast body of enpirical 
Sd ?heoreti^!l^ork i-read^ done, and we .must synthesize these theories 
I Sd iTi^Ti minifying pSadi^ that will both direct pur rese-c^ and 

facilitate our rapport with rural develojsnent practitioners. Research „ 
I on the evaluations rural development efforts can be considered as hav- 
ing three purposes: 

• Yl) Accountability: providing data to indicate whether the rural 
^ develSent e^foi^s are consistent with the enabling legisla- 
tion, the Rural Development Act of 1972. 

(2) Helping practitioners: providing data and predictive knowledge 
^ ^ tha? wm enable those in charge of the rural developn,ent ef- 
forts to do their jobs more effectively. This requires a con- 
JSiLg ra^rt beJween researchers and practitioners, including 
re^Sfd attempts to apply existing scientific knowledge and to 
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identify research problems that e^e relevant to the decisions 
to be made by the rural developnent practitioners. 

(3) Scientific growth: Improvement of the quality of ^Jj^ *f 
data, and methodologies available to researchers; that is, 
- takiilg steps toward further perfection of our paradigm of 
imreuL development. 
The primary focus of this essay is on the seconded third of these 
purposes. • j 

The Essential Role of a Research Paradigm 

In this conte^ct, I use the word "paradigm" as ^homs Kuhn (1^0) 
has used it in his Important, treatise. The Structure of Scientific Rev- 

S^tions. If we vieTS^elves ^-^-^^^^^^^^f ?/re"that con- 
rural dev eloptnent , thenShe relevant P^^^m Is taken to _ be that c^^ 
n+trifttion of beliefs, values, 'techniques, and models that underly and 
lenf alense oflohesion and ^ motive force -^-^^^.^^.^jf,-' 
vidua? efforts in this\dooain of our professional lives. A scientific 
SraSg^ seems to be co\ceived first as a set of cc«nmonly held^eliefs, 
vSuesf and concerns such as a perceived social °^^°^^°^Srier;ie 
Tn its embrvo stace, a multiplicity of mutually competing theories vie 

Sr'theTl-e ST^i.ence,|n the P-J^«-,,^f,^CS^ 
bom paradigm is embraced by all, or virtually all, of J^^^^f^^^ 
cZ^ty tfwhich it pertains. ^^^^^^ 

i^verification through the^ process Kuhn calls "normal science J^ic" 
incSIes a "moping ^P" of tL untidy aspects of the paradigm. Most of 
Sat v^ c^^eaJch falls in the category of normal science. 

Contrasted with this variety of endeavor is "revolutionary Bjience," 

pxacea cj.iiBOJ.ocLi. revolutionary new paradigm is, 

:SJonf i^lh^fea!^ ?ha n^w ^S'a^i^ Replaces the -old when it can 
iSvo ttf p^oble^ lerdl,« to the crisis, thus providing, a theory-based 
place- for the previously unexplained phenomena. 

Tn the context of rural development, an example of a (rather vsgue 

-^Lriais wsr- s^^sSit-rfor 

improved quality of life. 

A^-r several years of mixed successes and failures, several lessoni' 
emerged,V'««»«^ ^° «^«'8e, from the experience. For one thing, it 
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became clear to several observers that the local coinmunity could be 
made to. feel worse off, rather than better, if their expectations 
were raised iinrealistically by wild-eyed promises by innocent, en- . 
thusiastic CAP personnel. Other lessons- were learned well, many 
applicable to rSal areas. But -as far as I have been able to discern, 
the multiplicity of experiences and lessons were not really fdUxve, 
they have not been woven into an explicit unified paradigm ^^1 
develoinent within, which future generations of researchers and Policy- 
makers and agency personnel could, more or less accurately. Predict . 
the outcome of various rural develoiment, stratagems. What yas needed 
then, and what is" needed now, is a theoretical basis upon which to 
build our collective knowledge about th^e "if-then's " of rural develop- 
ment, a systematic and explicit framework within f ^3^^^. 
improveupon our ideas of the causal relations and prooabilistic state- 
ments thafconstitute our rather rudimentary theories of rural develop- 
ment • t 

The concepts underlying the Title V activity could be viewed as a 
proposed new paradigm, in very rudimentary form. What we are now test- 
ing is the proposition that local rural develbpnent agents, in close 
cooperation with researchers and extension personnel at colleges^ai^^d^ 

universities, can do a better job of -^Y"^/^"^, t'lnU^"^^^^^ 
pen than the CAP agencies were able to do. Some^day in the future, 
hot)efullv there will emerge a mature and fully developed paradigm, 
contfSiS Se Li^tages from both the CAP and the Title V approaches, 
as welll^ from all other experiences containing significant lessons. 
This body of theory and knowledge would serve, even better than any pre- 
^ous efforts, to^nderstand the causal relations underlying the process 
of Tvoral development. 

It has been said that progress is constituted in making all^^the 
errors as rapidly as possible. Bacon i^ quoted as S^Y^^e.^^^^'. gf^^ 
emerttes more readily from error than from confusion (1970, p. iOj. 
ISt o^ 7^rs will'^not teach us anything unless we have some logical, 
systematic, theoretical basis within which to asstoilate and interpret 
the errors'in the context of the successes and other relevant data. 
Kuhn says ". . .a paradigm ie pre-requisite to perception itself. Wha^ 

sees de^ends^pon Sat he looks at and also upon what ^J-P-vio-s 
visual-concepSal experience has taught him to see. In the f^sence of 
Jxlh trainir^ there can only be, in William James* phrase, a blomin 
■ buzzin' confusion (Kuhn, 1970, p. 113). 

Is it really necessary to deal with theories and', more broadly, 
with jLatof t^ To reseSch on rural development? No, it is\not essen- 
S2, S we^e content to wallow in confusion for the next several years. 
It is not esSntial if we are willing to come up with a cumulative zero 
addition to SskbZiity to make systematic predictive statanei^t regard- 
ing what will and will not work in his continuing effort to^ Improve mean- 
iSlfSly Se quality of life, incane, employment and general level of 
o^Siities'in r^al areas. Clearly, no researcher je^ts to^e fac^ 
vSh the prospect of making an accounting for his stewardship with nothing 
S show for the time, .effort and dollars that have 

cause We all want to be as useful as possible in our individual and ^ 
collective endeavSfs. How, then, shall we proceed? I do not pretend to \ 
hSe aS the Sswers. Hopefully 'this essay will focus our attention more 



Q019 



15 



clearly on the' questions, and propose aome ideas that will prove fruit- 
ful in- this conference, as each of us tries to come to grips with dif- 
ficult problems of designing our research strategies in the most 
productive ppssible way. 

What Ought to be Measured? 

What kinds of things should be measured in assessing the impact of 
a rural develoiment program? How should the results of this assessment 
be used? VJhat kinds of analysis are needed to provide the most useful 
perspective for interpreting the impact estimates? 

How to Shovr a Program Has Been Successful 

Elinson (1972) has, tongue in cheek, provided a list of things, not 
to do4f an objective measurement of program impact is desired, iiie 
following is a paraphrased version of his list. When one wishes to show 
that a program has been "successful," the evaluation should include the 
following: * ' 

1. A control group should not be used, or if.it is included in the 
survey, the data from the control group should be ignored in 
doing the \analysis. 

If a control group is used, it should be selected purposively, 
or matched retrospectively so that the data will look' as favor- 
able as possible to the.program. 



2. 



^ Effort (input) variables should be used as criteria of success, 
rather than effect (Impact) variables. Dollar volume of pro- 
cram budget, number of persons contacted, tonnage of educational 
material distributed, and other such effort variables are frej. 
quently used in lieu of any direct measure of the impact of the 
program. 

U. The evaluation should be done preferably by those 

the program; or at least by persons directly answerable to those 
in charge. In this w^,- the evaluation can be steered away froja 
detection of any dimensions of well-being that are^being adversely 
influenc^ed by the program; also, the analytic^ and survey tech- 
niques can be better controlled in the desired direction. 

5. The resuivts should not be subjected to peer review nor should 
they be published in the scientific literature. Ratjer, a con- ' 
ficlentialWeport should be issued for limited distribution. 

\ 

The Domain of Evaluation Research 

Considerable dilagreement is found in the literature regarding the 
appropriate and legitimate domain of program evaluation. On one^hand, 
SSa^ (1^3) a?gu^s strongly that program evaluation should be keyed 
rarerlfoseS io assessment of the effects of interest to Pr^gr^.^^-^ 
istrators. Her contention is that much program evaluation research in the 
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past has focused on testing of hypotheses interesting only to the 
Ivaluator, and of little or no interest to the program administrator. 
She argues that the testing of null hypotheses should not be done in 
program evaluation; rather, tests of significance should focus on com- 
parison of relevant alternative hypotheses posed by the program People. 
She proposes that the data gathering and analysis processes should -be 
intimately. tied in with the value judgments, preferences, and expec- 
tations of program decision-makers. She contends the decision as to 
the type of measurement to be undertaken should be based on tw? ques- 
tions: what is at stake in the program decision, and .what axe the odds 
of the various alternative contingencies actually occurring. 

Clearly this rather tightly circumscribed kind of evaluation effort 
is desired by some program officials. One would hope^ however, that this 
approach would be accompanied by a broader kind of inquiry, not limited 
to the hypotheses interesting to the program personnel. In contrast^to 
Guttentag, and in the interest of science and society, researchers should 
be encouraged to ask a broader range of questions,, sane of which would be 
characterized by program administrators as irrelevant, or perhaps irrev- 
erent. 

Arrow's conceptualization of social issues (1963 and 1967). seems 
highly dependent upon the implications of alternative choices, in the 
broadest possible sense. Arrow views each policy alternative^as^a vec- 
tor, composed of many different dimensions. Researchers should be en- 
couraged to probe many dimensions of well-being.. Some program adminis- 
. trators would- restrict our range of inquiry to those dimensions likely 
to e^ibit a beneficial effect fl-om their/program; they would feel more 
canfdftable if we would ignore the* spin-off or side effect d^ensions 
of w&ll-being also influenced by the program, either in the short run or 
the long nm. , ' 

In setting out to 'evaluate a- rural development progrwn, ancillary 
analysis is usually needed to provide a realistic perspective. A program 
evaluation ideally includes four stages: (l) context, (2) input, (3) pro- 
cess, and (U) output: end-product or impact estimation. , 1 

"Coiitext evaluation" would be used when a project is first being 
- planned (Stufflebeam, I968). Its major objective is to. define the envir- 
onment whire change is to occur, the current trends, the environment s 
unmet needs, problems underlying those needs, and opportunities for . 
change. This information leads to the establishment of program goals 
and objectives. 

i 

"Input evaluation" would identify and assess resources and appropri- 
ate strategies to meet these goals and objectives; it would also consider 
alternative program deigns . 

"Process evaluation" is designed to reveal essential features of 
program Implementation which contribute to the success or failure of the 
program. Process, anaJ^sis is intended to identify or predict defects in 
procedural design or its Implementation by monitoring potential barriers 
during program implementation. Process analysis must, according to Sam 
teadley (197U^, be performed during the prpgraia's implementation. The 
data\to be coliectedTdepends on the program involved and the researcher s 
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selection of a social science model of change processes. "For example, 
£ s^iological model for institutionalization of services alerts the 
^elearcher to data dealing with legit iination, cooptation, and the appli- 
cation of social control mechanisms." 

Key process variables that could be monitored are (l) the develop- 
:aent of cLmmication patterns, e.g.,' how information i^J^^^^.^^^J^^^^ 
received between clients and program st.^ff ; (2) the emergence <fj^^^^ 
agreements regarding program intent and mechanisms for social control, 
e rules, guidelines, policies; 3 the development of logistical 

inSiacuS pattern^ feading to progra^n legitimation, /^ocess eval- 
u^tion is initialed a^er a designed course of action- has ^een ap^oved 
and implementation has begun. Analysis of process examines the P^th 
followed from the initial plaiinlng stages to P^°6r«^ !^^J3' 
when attention is focused on process as an entity, ^i^^^. ^^^J^-^f 
are an integral part of the overall evaluation, since observing the in- 
teraction of the new with the old is a primary objective. 

Standing by itself, process analysis is not intended to evaluate the 
success"r?ailL of a'program. Output or ^ -ttoation is ^e^ed 
for that purpose. By examining the procedures followed, process ^^f^ 
can proviinvidence as to ^ the program may have f 
This information is essential for improving program ^^.^^S^^oS^^S 
± e for effecting process control. Thus, process analysis, along with 
biniex? Lf Sput^^ should ordinarily accompa.^ the analysis of 

program output or Impact . ^ - 

"Impact estimation" is the most obvious type of analysis, the type 
most people seem to think of when program evaluation . 
suite its pre-eminence, however, ianpact estimation is no more Important 
?o a thorc^h program Evaluation than any of the other three types of 
analysis. 

Multiple regression is often used in the estimation of the^lmpact of 
the program, whUe statistically holding other variables constant. Re- 
gardlesf S'trtophistication of the statistical methods --d> Jj^^^^' 
uSess the relevant data are collected in a jwdiid manner, the Results 
Si be uselifr%e .question of relevanc^^Si^s °- ^he Paradi^^under- 

lying the study. Unless a set of data ha^a ^^^fo^t^^^^.^n ' 
o^erln evaluation model, chances are the data isn't ^^o^th^ collecting . 
The paradigm should dictate -the data to be collected, and it should be the 
basis for interpreting the findings. 

Validity of the Data 

. 'campbell and Stanley (I967) list an ^^^^^'^^y °J J^^^";. 

countered in retaining validity in measurement devices. One of the proD- 
1'^ Sey c?te isIS of the Lactive effects of experimental arrangements 
"SLh^uld preclude generalization about the effect of the experimental 
vSablfupon persons being exposed to it in nonexperimental settings. 
tSs is a^ example of the problem of external validity, or representative- 
ness Jf Se ^^Sts to a broader population beyond the- scope of the^sample. 
For exL^le, when one county or location is selected for a rural develop- 
ment eSrt how confidently can we apply the lessons l«f "^^.^J. f 
locations lA the same state or in other states? External validity is 
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contrasted «ith internal validity, vhich has f ^^^^^ interpreta- 

tion of the effects found vith reference only to the sample. 

In addition to the selection or developuent of ^PPropriate instru- 
ments to collect data, it is iinperative that the s^ey or other data 
coSection process he designed properly, so that external YJ^^^^ 
hp achieved One of the most comprehensive treatments of this subaect 
is SverS'caShen and Stanley (196?) . Webb, et; ai (l966) presents 
a vf r^ Sseful nst of potential problems and pitfalls in doing survey 
releSc^ Hardin (1971) discusses- a wide variety of approaches to de- 
^el^en^ oJ^n^rol ^oups . Some rather "^^^l and interesting approaches 
Se^gested. Elinson (1972) has presented a brief and rather critical 
Te^i rof the scienUfic'and statistical standards usedj^n st^ie^o^ ■ 
determine effectiveness of social action programs ^. J^^^. ^^„''Jt3j^^ * 
He claims very few (he counts only 10) studies over the past ten years 
have met his rather stringent criteria for rigor. 

Webb and others (1966) discuss several factors that have been tai^ 
to reduce the internal validity of the measurement operation. For exam- 
IhS Ust the "guinea pig effect"; this is due to the awareness of 
Sl^<. tSted Another xE-dblem is what they call "role selection, the 
^acftSf fo^e ^fpoSeSs tend to JP^^^^^f^^^^ 

^^^^^ ^^^^^J^ 
S bSSS! ^Lr^SfelS^^^^^^ 

A ^rsistent problem in obtaining attitudinal data is one of res^^se 
, sets. Sgnine Sd'lcanl^ (1973), for example, found a serious probl^ 
of acquiescent response set among ^^^^^ -^^^^^^^t ^It co^erse 
son tends to respond favorably to both a proposition and .its converse . 

Measure of HaE^^ 

Miller (1970) has. ccmpiled an impressive array of survey instru^jts 
that ^^be ierio meas^ various aspects of well-being The q^^^^^ 
aSressJd here is not how to "measure happiness or life-satisfac^ 
Aether such measures ^ appropriate for the evaluation of rural devei 
opment programs . 

Soiae societies, particularly Western ^^sojieties, ^^^^^^^J" J^^J^^" 
•ized-a?Seing on a "Hedonistic Treadmill." That is, as i^^^^^^^^.^f 

urement for program in^^act* 

TPo«4.^v»nr. (ion) found a positive relation between "hap- 

Eleven of these surveys 7>ere done In the "n"®*/*^*^' ™^ eleven 
acne in other countries, l-^f feS o^^^^ 

'X^U,A\^liuT^ ^tl^^ruT. «hen you c»pare rloh 
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and poor countries, or higher and lower income situations in a given 
country at two different times, the happiness differences one might 
expect on the basis of differences in income do not appear. 

' Easterlin offers an explanation for this paradox. He suggests the 
answer lies in the way people form their value judgments regarding their 
criteria for happiness. "The satisfaction one gets from his material 
situation depends not on the absolute amount of goods he has, but on how 
this amovmt compares with what he thinks he needs." Even within the 
united States, he points out, the great majority of Americans have a level 
of living that would have been considered wealthy two centuries ago. None- 
theless, the typical American today does not consider himself wealthy by 
contemporalV standards. Economic growth and rising incomes are accompanied 
by an upward, shift, in perceived needs, which tends to offset the positive 
effect an increase in income would have otherwise-'. This syndrome is caUed 
the "Hedonistic 'Treadmill." 

Strumpel (1973) comes to the same conclusion, while drawing a fine 
distinction between econanic theory and psychological adaptation level 
theory. The latter theory assumes th&t man has an, almost unlimited capa- 
city to adjust to reality, his yardsticks for judging his situation. As 
the environment becomes more pleasurable, subjective standards for gauging 
pleasurableness will rise." 

This concept has staggering Implications for the measurement of satis- 
faction associated with programs. "Suppose, for example, there are^two 
groups of persons, essentially equal in characteristics except that one 
group is about to enter into a new intervention program bringing them a 
far higher level of consumption of some specific goods or services. \Thls 
could be: for example, the Congregate Meals Program for the^Elderly.^ Let 
us consider three points in time. At time one, just prior to the^initiation 
of the program, both groups are observed in a sample survey. At time two, 
the same individuals are re-interviewed using essentially the same survey 
instruments, and it is ascertained that in this very short time period the 
participants have become significantly, happier than they were in the first 
survey. The obvious conclusion is that the program induced this increase 
in happiness. (Let us leave aside for the moment -the way happiness would 
be measured.) 

Now suppose the Hedonistic Treadmill comes into play, ahd the program 
participants begin to aspire to even higher levels of these/same or .perhaps 
different goods and services than the ones provided by the yprogram. Con- 
ceivably, if these new and additional goods and services are not forthcoming 
as desired, the participant group could sink to their previous level of. 
happiness, so that in the long run both the participants /and the non- 
participants are equal iri this .regard.. In the brpad- historical and cxatural 
perspective, this seems like a distinct possibility. / 

This being the case, it seems imperative that progrmns be evaluated 
not solely on the basis of perceived changes in satisfaction or happiness 
(however measured) but prltaariiy on the basis of changes in objective con- 
ditions-e.g., availability of services within 30 minutes, sanitary water 
supply, increased employment, higher incomes, more equal distribution ol 
Inc^e or wealth, etc. In other words, the goals of the rural develoiment 
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programs should be stated in terms of objective measurable outcomes, 
vhich are explicitly recognized to be related to happiness^ or satisfac- 
tion in one way or another. A program which is effective in improving 
one or more of these con^litions would not ^necessarily be judged unsuc- 
cessful," just because the clients remain (or become increasingly ^ un- 
happy or dissatisfied. Dick Hoke has put it rather well: An under- 
nourished person who enters the program constantly hungry and. then 
regularly receives three meals a day is not being. poorly served merely be- 
cause he remains dissatisfied due to a- continued craving for cheesecake 
and chocolate eclairs:'. .it is possible in most value contexts to distin- 
guish between a perceived good arid a real good, with the latt|r being Jin 
principle) objectively determined by the judgment of the proverbial scien- 
tific- investigator ' He goes on to say that th^ happiness of the client 
population should not be ignored,, but that this should not be the primary 
basis of assessing the luipact of a program (Hoke, 1974). 

Obviously, we as researchers should not ignore the subjective meas- 
ures (values,' attitudes, contentment, etc.). Far from it. As Wilkinsoi) 
has pointed out (in his excellent review of an earlier draft of this 
essay): 

"...It is possible to measure attitudes and attitude change 
• without making normatively hedonistic assumptions . Measure s 
of alienation, aspiration, community identity. and general 
•climate of opinion' can be very ustful in assessing dmpacts. 
The error would be in relying entirely on these. I would 
■ suggest that the gap between objective and perceived changes^ 
in .status (e.g., between 'absolute' and 'relative' depriya- 
tion) coold tell much about quality of life, unless we are. 
assuming that the onljr real deprivations are absolute, (I m 
not able to assume this.)" ' 

I concur with this suggestion up to a point: We should not shy away 
from making normative judgments as to the human value relevance of the 
^ dimensions we select for. measurment in an evaluation study. In fact, we 
should make this selection within an explicit paradigm. Each dimension j 
should be clearly identified as either a means or an erid within the over- 
all model of rural development and individu^/social wll-being. 

One reason why the subjective data must be collected in an evaluation 
study is conrpelling:- to i«-ovide a basis for validating the paradigm 
the objectivVmeasures. It is possible (hopefully unlikely!.) that th^ 
subjective data would indicate the people' con;jider themselves^worse off , 
while the -objective- datii all indicate they should be better o^^^^^' J^;^* 
were to happen, we would be obliged to take a.hard look at our objective 
meSures aSd o^ idea of what constitutes "progress" in rural development;, 
our paradigmAWDUld surely have to undergo major surgery, including both 
■ axBputation and organ transplants. Conversely, one would hypothesize that 
i^rovements in the objective measures should be positively correlated 
with improvements in several of the subjective itans, such as those reflect- 
ing perceived satisfaction, relative deprivation, alienation, etc. 

t 

Organizing the Data 

Once the decision has been made as to the appropriate data to be col- . 
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I'ected 'arid the site has been selected, we are still faced ^^^^f.ff'- 
.lem of collecting and organizing all the necessary data. J^e Day has 
sxiggested a matrix for use in assembling the data. 



Administrative Structure 



* 

If J were'^to venture a guess as to the aspect of , rural dev^Lo^ent 
research that is most critical and most likely to be the limiting factor, 
causing failure of the whole effort ... my answer would be the adminis- 
Sati^f structure. The level of funding of rural development intervention 
programs and evaluativfe research are obviously miniscule in relation to 
?he absolute need. However, I view inadequate funding as the pcond most 
significant barrier to effective rural development research. Even with 
adequate funding, our ef-fo^ts will pfove to be futile unless we a)^VQid 
scattering our research refeirfces tVo thinly, (2) learn ib prganize our- 
selves into effective research efforts with a well developed research 
paradigm, and (3) remain en rapport with the local community and th^ rural 
development practitioner. 

A corollary of inadequate funding is too low a level of the quantity 
and quality of the scientific manpower input to the research. .The summa- 
tion of several fractions of a man-year, often adds up to a ^erb, in terms 
of' creative research. Sismondo contends that all excellent qu^ity re- 
seafch. efforts he has seen have been based on at least one intellectual 
giant (who can grasp the relevant theories) rather than 4 or 5 rookies. 

The problem of critical mass is nowhere more problematic than in a 
relatively new and uncharted line of research The «f ^^^^^^Jf ^33°;. 
an appropriate theoretical paradigm has been discussed. It i^ f^st essen 
tial that each researcher in charge of a significant P^f *i°n ft^ ^^^^"^ 
ation be given enough freedom ftom his or her other duties, and f^^^^^. 
with his professional peers and persons from other relevant ^i^° JPff 
so he or she can internalize the necessary P^^^^6",'^f 
niques neededlto gather and analyze the data effective;y. ^What is likely 
to happen is that too many researchers will be spread toe thinly, to be . 
able to make the intellectual investment necessary for a really creative 
research effort. This is a challenge to the research administrator; I- 
would like very much to be proven wrong on this prediction. 

The second critical area of administration is the need to maintain 
rapport (including respect and- trust) between the ^^^^f °ifj' 
tended audience. The-researcher needs to be in, rapport with the policy 
makers and agency personnel and other practitioners at.the grass roots 
level, both at the beginning of the study (so the correct research^aues- 
tions are addressed) and after the study is completed the finding s-jfaust 
be communicated to the users in a way they can understand and lappljif to 
their on-godiig operations. During the first stage of the research--the 
context analysis-the researcher will inevitably cc^e up wlth^a 1°* of 
S?a that will be of great interest to the program administrator and other 
^IctiJionSs irivoi;ef in the .rural development, effort. These data should 
be communicated to these potential users as clearly and as rapidly as is 
feasible within the constraints of good quality research. 

> 
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The roles of the^researcher and the practitioner are highly com- 
plementary and frequently overlapping in the task of analyzing and 
helping to resolve problems of rural development. Researchers can 
facilitate- this development by providing timely and lucid knowledge. And 
by being actively engaged with practitioners in providing guidance for 
action, i:he researcher receives a feedback of additional research prob- 
ieans and exceptional observations which can spawn additional hypotheses 
(Madden, 1970). 

Th** finest example I have seen of maintaining continuing rapport 
with the user-clientele is the' work of Sergio Sismondo fn New Brunswick, 
Canada (1973r, 1^3b). His operation included both research and opera- 
tional components, constant-ly-'in .contact, with each other and with the cli- 
entele. Their results in terms of actual accomplishment of several key 
aspects' of rural development is astonishing— increased income and employ- 
ment, increased political awareness and participation by the masses, de- 
velopment of new industries- employing hundreds, of formerly welfaxe- 
-dependent'5>ersons, etc. Sismondo is absolutely convinced that the success 
of the program (called New Brunswick NewStart) is critically dependent 
upon maintaining rapport between the research team and the community, par- 
i:icularly the. local power elites, agency personnel, and entrepreneurs. 

The Task Force on Rural" Development , within the National Academy of 
Science, has recently finished a report entitled, "Guidelines for Manage- 
ment and Design of Rural Development Research." This report emphasizes 
the need for a thorough study of .the- way in which rural development re- 
search may be most effectively and efficiently organized to accomplish 
the tdtlmate objectives of rural development (Eberts, 1974). If this 
study Is funded, it could provide an excellent addition to the development 
of a truly effective paradigm for rural development research. 
-% ■ . • 

Comments on the Evaluation Plans in the Proposals of the 
Twelve Northeastern States 

» 

Evaluation plans from 12 Northeastern states were sent to me as part 
'of the background reading for the preparation of the essay. The plans I 

received were abbreviated and, in some cases, possibly preliminary. 

Therefore; I o-ealize that the impression^ gleaned from, those materials may 
' be erroneous, far out of date with current plans of the respective states. 

With this admission of fallibility, I will now proceed to 'summarize the ^ 

major point in the- evaluation components of the proposals. 

Five of the 12 states have selected a particular ^service as the focus 
of the ^ural'-deveiopment effort. This decision was made apparently on the 
advice' of an appointed advisory council, or at least with* their acqules-^ 
cence. In two other states, economic growth (income and/or employment) is 
'the focus. -In ohe state the emphasis is to be on some topic of primary • 
interest to environmentalists. In the remaining four states (Massachusetts, 
Haimshire, New- York, and Pennsylvania) this, decision has been postponed^ 
until -the interests and priorities of the target area and/or its represen- 
tatives have been determined. 

1^ The proposals are, for* the most part, not clear as to whether a base- 
line" survey >rill he done. The states seeJn about evenly split between those 
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that intend to determine program priorities via surveys versus those that 
Choose to rely on the advisory councils.- A household survey will be done 
>n JorJvl^Slia for example, tb provide data to be used later in estion- 
SiS prS^l^pac^/rSl^her certain elements of the context data and 
?o StTSing on the values and priorities of the population in the 
s?ud^^ea! pShaps other states will be doing a similar study, but this 
was not clear from the material I received. 

In k of the 12 states, the evaluation plan stated explicitly that 
the sSte ol iScal advisor^ council will have the responsibility of evai- 
uatiS the ^al development activity. Given that these cou?icils are com- 
S^se^almosnxclusiveirof very busy executives, it seems clear that only 
TTery perfunctory "evaluation"^ will, be perfomed. And given Jhe^small 
amount of money allotted to sane of the states lor this ^i^le V activity, 
perhaps no more than a cursory 'evaluation is warranted. Anyth^ more 
wofound might be an utter waste of everyone's time and a chunk of th? 
ISued bud|et.-a little like doing an extensive study to determine whether 
aticket of red ink dumped in Lake Erie would cause Niagara Falls to turn 
redT pLf orthroutput of the Title V pilot ^tudy,/nation.wide should 
ITb clSr signal tb .Congress and the Office of Management and Buoget re- 
gLdiSThe minimum criS^al mass of research and °P-^^^"f ^f^'r^^:^^^ 
which w effort would be a total waste. Seme of the 12 state proposals _ 
leave m^^with the uneasy feeling that they fall below this minimum. 

* Another impression gleaned from my review of /the^state PJ°P°^f ^ 
+h^t nearlv all of them lack several, of the key ingredients of good sclen 
Sfic ~h' Perhaps ,his^ impression is incorrect, duetto 
Intemretation,- or maybe the state proposals are out of date. More than 
nS ^he researChS,. involved cannot be held responsible for the seem- . 
in^ to^dequacies^because several are spread so ^^^"1^ f JSl^^ 
and various rese^cTi responsibilities that they cannot be expects^ to do 
TorlS A Job oh their .«Sal developnent research and^ program evaluation. 
NonetSlesi, as they now stand, the state Pl--6enerany lack a clea^ 
recognition of the need to build a paradigm encompassing .he massive Doay, 
S tSory S^td empirical research in the literature on human ecology, re-/ 
gUS^oSics! public finance, social stratification (P^^^icularly^on . 
vertical social mobility) demography, political science, ezc. The vast 
intell^tual resources of social, science need to be tapped. A major pb- 
s?lcie ?o ov-ercome in such imLLtildiscipline teamvrork is the "Tower of 
Blbei" s^drome >merein the specialized vocabularies of^the v^W 
ciplines prohibit effective communication' essential to the construction 
of a unifying peo-adigm. ^ 

■Another crucial -element typically found lacking in the ^^ate plans is ' 
' a systematic mechanism for providing continuing rapport between the re- 
42cher8 and (l) the clientele population, and (2) the rural development 
p^^^Uti^nSs.'.Sismondo (1973b) has suggested a B-era pol resear.ch 
Paradigm which -makes explicit the circular flow of information between _ 
?S reSarchers and iihe significant others f %^J^^^;;S°?;'^f^^jr 
This is such an essential aspect of the overall strategy that it can be 
Snored at the peril of the success of the rural development research. 
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Conclusion 



These comments axe offered here mainly to facilitate further dis- 
cussion in the .various work sessions. Hopefully the concepts presented 
will illumine our efforts to make a truly mePOiingful input into the 
knowledge base_,^ the theories-the research paradigm--at the heart of 
the" scleroe oT rural development. 



0029 



25 



* 

References cited 



1. Schumacher, E. P. Small Is Beau tiful - Economics as If People 
. Mattered . New York: Harper and Row Publishers, 1973. 

2. Jansma, Dean; Gamble, Hayes; Madden, Patrick, W^land, Rex. 
"Rural Development: Problems and Prospects." (Unpublished paper 
prepared at the request of the Literature Review CMinittee of the 
American Agricultural Economics Association; to appear in Am- Jour- 
nal of Agri, Econ .) Januarj' 197*4. 

3. Wilkinson,, Kenneth P. "Social Well-Being and Rural Community 
Develotmeit: Issue.6 in Palicy and Theory." Department of Agricul- 
tural Economics and Rural Sociology, The Pennsylvania State 
University, unpublished manuscript, (delivered .at 19714 Rural 
.Sociology Meetings in Montreal), U8 pp., January 197tv. 

U. • Sismondo, Sergio. The Measurement of Develoiaaent: Selected. Findings, 
f or Rural Conlunitles in Kent County . Richib^to Brunswick, 
Canada: New Bruniwick NewSt art , Inc . , Report No . R-73-115 , 19T3 . 

5. Sismondo, Sergio. Social Indicato rs for Policy Research^and Danocratic 
^ Action: A Pa^adi^ and Some Examples , ^^^^^^^^^'^l ''^^^^^ T 

Canada: New Brunswick NewStart, Inc., Report No. Rr73-119, ■'•9^:$. / 

6. Warden, Roland L. "Alternative Cowmmity Paradigms/- Paper presented 
at Eastern Sqciolbgi^a Society seminar, April b, 197*4. 

7. Kuhn, Thomas S. The Structure of Scientific Revolutions. Chicago: 
The University of Chicago ^ess, 2nd Edition, 1970. 

8. Elinson, Jack.- "Effectiveness of Social Action Programs in Wth 
and Welfare," in Carol H. Weiss, Evaluating A ction Programs, 1972, 
TJP. 29^-299. 

9 Guttentag, Marcia. "Subjectivity and Its Use in Evaluation Research," 
Evaluation , Vol. l^-^o. 2, 1973, PP^ 6O-65. 

10. Arrow, Konneth J. Social Choice and Individ ual Values. New York; 
John Wiley Se Sons, Inc., 2nd Edition, 19^3. 

. "Public and Private Values," in Human Values and 
* Economic "Policy T ed. by Sidney Hook, New York University Press, 1967, 
pp. 3-21. 



* In the order in which references first appear in the text. 

S 



0030 



' a6 . 

I 

12. Stufflebeam, Daniel U ^'Toward a Science of Educational^Evaluation," 
Educ ational Technology , J\xly 30, 1968. 

1^. Leadley, Sam M. Preliminary manuscript on program evaluation. Depaxt- 
ment of Agricultural Economics and Rural Sociology, The Pennsylvania 
State University, 197^. 

lU. Campbell, Donald T. and Stanley, Julian C. Experimental and Quasl^ 
Experimental Designs for Research . Chicago: Rand McNally & Co., 19t>7. 

15. Webb, E. J.; Campbell, D. T.; Schwartz, R. D.; and Sechrest, Lee. 
Unobtrusive Measures . Chicago: Rand McNally & Company, V^. 

16. Hardin, Einar. . "On the Choice of Control Groups"; Paper presented 
at the Conference on the Evaluation of the Impact of Manpower Pro- 
grams, Columbus, Ohio, June 1971.' 

17. Pignone, M. M.i^SND., and Scanlan, A. S., SND. Participant Analysis: 
Congregate Meals for the Elderly in Luzerne County, Pennsylvania. 
Paths Out of Poverty Working Paper No. 6, Department of Agricultural 

• Economics and Rural Sociology, The Pennsylvania State University, 
November lSr73^ ^ 

18. Miller, DelbertC. Handbook of Research Design and Soci a l Me asure- 
ment . New York: David McKay Company, Inc. , 2nd Edition, 1970; 

19. Easterlin, Richard A. "Does Money Buy Happiness?". The Public Interest, 
Winter, 1973* PP- ^3"^' 

20. Strumpel,^urkhard. Economic Well -Being As An Object of Social ■ 
Measurement . Ann Arbor: Institute for Social Research, 1973- . 

21. Hoke, Richard. Personal correspondence relating to r.eview of a 
manuscript, 197^. . 

22 Eberts. Paul. "Guidelines for Management and Design of Rural Develop- 

• ment Research Unpablished Report of the Task Force on Rural Deyel- 
opment. National AcadenQr of Sciences. Department of Rural Sociology, 
Cornell University, Ithaca, New York, August 197'<-. 

23.. Weiss, Carol H. Evaluating Action Programs . Boston: Allyn wid Bacon, 
Inc/., 1972. '• , 

2U. Madden, J. Patrick. "Social Change and Public Policy in Rural 
■ SericI: Data ^md Research Needs for the 1970's," American Journal. . 
of Agricultural Economics , Vol. 52, No. 2, pp. 303-311*, May 1970. 



/ 



« 

0031 



27 



EVALUATION NEEDS UNDER TITLE V 
OF THE RURAL DEVEIOIMENT ACT OF 1972 

Howard C . 'Tankersley 



Program Evaluation - One Vlev 

'proaram evaluation is accomplished for one or more of three tasic pur- 
poses (fHo^l^e program adjustments to increase the effectiveness of 
?Se program underway or to increase the probability of success with the 
neJt eS Te undertake; (2) to determine the "vorth" of staff numbers or 
?o lu^ti^Tsalary adjust^nts, promotions, terminations, and/or other per- 
son^S aSionsTl^d 3) to justiiy expenditures for programB to those who 
avSd or administer funds. ES and GSRS are interested xn evaluation to 
i^ifV expenditures and to make program adjustments. We have little in- 
Sesf in^ali^Son to determine "^orth", of personnel. This ve consider 



State business , 



The dictionary defines Evaluate: to ascertain t he v,glue_o£. A search 
of evaSltiormerature reveals that evaluation requires three basic func- 
^L^i^c^i my bfa^^ as three stages of the evaluation process: 

1. Documenting - furnishing evidence of needs, plans, procedures, 
•inputs and outputs or results. 

2. Arraying evidence documented in a logical or rational display. 

3. Appraising - setting value on the 'evidence arrayed. 

Many evaluation systems have been described in the literature but, 
T asser^That all fit into a continuum containing four fairly distinct 
JvSs Tie continuum runs from Cost/Benefit -Analysis, vheredonar costs 
aTb^nems are measured on cardinal scales, to Classical Operations 
S^na^emeS vhSl a^ordina^ scale of costs is displayed against an ordinal 
scaS of ef^eSs or achievements. Cost/Benefit 'Analysis is ^ways done in 
a fo^l sensfara is documented while Classical Operations Management is 
ofJeS^ed Sthout documenting anything in writing and is carried on xn the 
Manager's or Supervisor head* 

Documenting and Arraying are always the P°J^i^i^i^^,°f .g^^^^f 
fessional or manager. The Appraising function is the responsibxlxty of the 
gofessional T^ager only in Cost/Benefit Analysis Appraisir^^^^^^ 
£ne by the decision-maker, those who award or administer funds, in the 

Tankersley is Program Leader - Rural Deyelopaent, Extension 
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otherl three types of evaluation systems discussed here. We must presume 
?hat thSe will be a direct correlation between the skill of the profes- 
sional or meager in documenting and arraying and the decisions reached. 

The four systems on the continuum are as follows: 

1. Post /Benefit Analysis - Dollar costs axe displayed against dollar 
benefits and Cost/Benefit Katio s axe computed. The ratio suggests that 
costs exceed benefits or benefits exceed costs ^"'^"^ 
the incidence of costs and benefits or who pays and who benefits. 

2. Pn.t. /Effectiveness Analysis - Dollar costs are displayed against 
a cardin al scsle of effects, kill' rat ios and °ther numbers games. Cost/ 
Effectiveness analysis. numbers games allow '^o'"?"*^^"" f J^!:!"^^ 
which allows the decision-maker to -det^^rmine for hnmself whether or not 
the effectl arrworth the-dollar costs. This type of evaluation sometiirfes 
defines the incidence of costs and effects. 

3. Program Appraisal - A cardinal scale of dollar or other costs is . 
arrayed- agaiSt achievement s. The incidence of costs ^J.^^"^^^^^/^^^^ 
SuSay Sfined. The ^decision.maker must determine for Jj^^^^f f 

of achievements and often he must determine J^\f 
costs. This system aMows for inclusion of social and political as well 

as economic costs* 

U Piocc-i..r o perations Management - An ordinal scale of social and/ 
or pollti^ costs Ze arrayed against ordinal scale f '^°"f^^!J^/°3'3^3 
^mical effects or achievements. The incidence of 

Sws defined. Examples of these costs and benefits would be. (l^ the 
cost of detente (political culpability) arrayed against the value oj^^ . 
?:Se (abfence of'^old war); (2) the cost of promoting - -f^f^^^/"^^^^ 
office of Deah (impact on the faculty) arrayed against the value of such 
promotion (new blood in the college). 

In evaluating Extension and research efforts effectively, we will 

r,rohi?v utilize Program Appraisal to satisfy the need for justif^ng pro- 

e^endii^esTo'd"isfon-m^^^ We wUl P^f f 
ApSaiSl and Classical Operations Management to determine the worth of^ 

staff members and/or to make program adjustments. 
Title V Needs 

Tn PVflluatimr Title V State programs, we face two kinds of needs. 
The fi^st SSe^to those provisions of Title V of the Rural Developnent 
of 1^2 iSich are different from' the Smith-Lever and Hatch Acts under 
Which moS eSensirand research work is accomplished in the land-grant 
cvitem We must evaluate whether or not we have met or exceeded the in- 

or eSeSSionfof the Congress in funding Title V. ^he se condone ed 
Is to evaluate program effectiveness. Meeting, these needs will allow us 
to ^ustSTor Lifto justify, expenditures and to dete^ine needed pro- 
grail adjustments - find out what we are doing right or wrong and indicate 
changes needed in the future, if any. 



0033 



29 



Intent of Congress 

An analysis of Title V and the legislative History of the -Rural De- 
velopment Act of 1972 reveals certain expressed or implied expectations oi 
the Congress. These are as follows: 

1. The audiences that will be served, are: units of general purpose 
government at the State and local levels; special purpose Hiulti-ourisdic- 
tional units of governments; organizations, of local^ people involved in 
development; and industries that provide, or may Provide, f Pl°y^^"^ 

the n^al community. I believe it is fair to ccnclude^tnat Congress per- 
ceived these to be the most important audiences to be dealt with in ruraJ. 
develoiment and that they were not being dealt with sufficiently at the 
time the Act wfils passed. 

2. Divisions of the land-grant university other than agriculture, 
engineering and home economics and institutions of higher education other 
thin land-grant universities will be involved in developing the rural com- 
munity if the extension arid research functions are to 

tivelj ^d sufficiently. I believe.it is fair to °°"<^^^\*5f ,^°"f^f 
held at least five perceptions about extension and research in the land- 
grant system at the time the Act was passed: (l) thejocus of ^hj exten- 
sion and research functions in the land-grant system^ has been confined to ^ 
the interests and disciplines most . common to the colleges of agriculture, 
engineering and home economics; (2) the disciplines and J^^J^^f 
traditional to extension and the experiment station are . 
appropriate to meeting the extension and ^f^earch^ne^ds of the audiences 
defined for development .of rural America; 3) Tit3.e V would shift the focus 
of l^d-grLt univSsity extension and research from agriculture engineer- 
inK and home economics to development of the rural community - to meeting 
the T^eds^f the audiences listed above; (U) Title.-V would^make possible 
e^ension Sid research work, appropriate to-th^e needs of the audiences de- 
fSed! In developing rural communities; and (5) research and extension work 
by other institutions of higher education is needed. 

3. Development needs of rural comraunitieB will be Identified by pro- 
fessionals and lay leaders in the States. Integrated 

search efforts to meet these practical needs will be projected in>the State 
Plans of Work which will be submitted to professionals and^lay -leaders for- 
ap^ovS. before being sent to USDA. L believe it is fair to conclude that 
?oSrIss perceived rSral development needs to be practical in nature, xhat 
S ough? to be identified, defined and articulated by people in the local 
ruSl communities, and research and extension efforts undertaken to address 
these needs. 

U. 'Research and extension functions of institutions of higher educa- 
tion will be co-ordinated, integrated and focused on practical problms 
expressed- by local communities. I believe it is fair to conclude that 
CoSress perceived a lack of co-ordination between research and extension 
functions in the land-grant university. 

' Educational programs (research and extension) will be co-ordinated 
with oiher ^Sai d^eSent efforts of the State, local and Federal govern- 
^n+R I believe it is fair to conclude that Congress perceived a lack of 
TorknLion bILeen t^ agencies of government at all levels in developing 
ruretl communities • 
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6 Title V is a small pilot effort of 3 years duration to test the 
techniques and organisation for rural development research extension. 
I believe it is fair to conclude that Congress meant %o test whether or 
not a federally-funded program of educational assistance to ^"^^1 commun- 
ities can succeed when it is: (l) administered in connection with the 
Hatch and Smith-Lever Acts through the land-gra^t universities' extension 
services and experiment stations; (2) when it is adined at solving practi- 
cal development problems of rural communities; and (3) when it involve, 
disbiplineffrom outside the colleges of agriculture, engineering and home 
ecorioaiics in the land-grant university and from other institutions of 
higher education in the State. 

I can assure you without qualification that all of us in ES and CSRS 
are cognizaiit of the low level of funding for Title V and the demands upon 
the experiment stations and extension services to supplement such funding, 
to support Title V programs. .However, if we have accurately identified 
the pSceptions held by Congress as the .reasons for including those provi- 
sions of Title V which are different from the Smithrl^ver and Hatch Act 
programs, we must conclude that Congress will not accept the argument ad- 
vanced by some that we have already proven, under the Hatch and Smith- 
Lever programs, the techniques and, organization effectiveness of the ^^^^^^^^ 
grant system to deliver rural development assistance. If this^is ^ correct 

St^ssment, we must also conclude that the J?^°^J^5Se ^epa^t! ' 

supply Congress and the Administrators in our universities and the Depart 
ment of Agriculture, .dil be taien seriously. We must, therefore, take our 
evSuatioS responsibilities seriously . If we have accurately assessed^the 
intent of Congress and the reasons for ^^^^^^p^S^f ^^^l^S 
latioh, we will need to document and array evaluation information related 

to the following: 

1. The audiences involved. 

* 2 The involvement of other institutions or faculty members from, 
colleges and universities other than land-grant. The extent and kind 
of involvement. 

3 The involvement of faculty from divisions of the land-grant 
university other than colleges of agriculture, engineering and home, 
econonics. The extent and kind of involvement. 

h. The role and involvement of the State Rural Development 
Advisory Councils.- The extent and kind of involvement. 

5. The extent to which the focus of the program is on Practical- 
problems of the rural community. The kinds' of problems and who identified 
them. t 

6. Co-ordination or integration of research and extension efforts. 
The extertt and'kind of. co-ordination or integration and the systems or al- 
liances which evolved to facilitate co-ordination or integration. 

7. Co-ordination between educational (rese.arch and extension) efforts 
and other rural development efforts and activities in the States. At least 
show that they were not in opposition to each other. ^ 

/ 
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A proper documentation and arrayal of infonnatioh -^^^^^^^^^ to these 

of the land-grant system to organize a program which wiH deliver rurax 
velopm^nt educational assistance effectively. 

ProRram Effectiveness ... 
, . Evaluation of P-^r™ f ~es.^ 

of the Rural Development Act of 1972. It is necessary development 
the effectiveness of the techniques employed i°^^^iJ^^",^"^^3„neSd the 
• assistance. The impact of the information f/^^^^^^S^ v are not 

i^nediate demands of Title V. Our program f^^^t^^^^'^/HSch Act. 

grossly different from our program ei forts under Sn>ith-Lever or n 

^unds. Therefore,,, any evaluation -^^^^^'^^^'^ ll^'c^ll^^^^^ 

be extremely useful in articulating P"^f ^' "^ifJ^^T""' Jf^rtHrc^un- 

subject matters dealt with in- all research and extension efforts in comm 

ity or rural development. / 
Evaluation for programSffectiveness ought to document 

ing kinds of information documented and arrayed: 

1 Inmts - a description of what was planned and what was actually 
• doS by resSche?s and extension workers in achieving or/pursu- 

ing the objectives. 

a. Research or technical^assistance provided - "Bhe kind' and to 
whom it was provided. / 

- V 

b. Information and educational as^istanc-^ provided - the kind 
and to whom it was provided. ' • / 

c . Organizations developed or assisted- - number and' co^JPOsition 
- SS^-up) of organizations created. Existing organizations 

assisted and kinds of organizational assistance given. 

d. Leaders identified, mobilized or assisted - numbers and 
composition. 

e The science man-years of extension and research ^^^^ given to 
Title V programs by all institutions of higher ^education in 
the State. / ^ ^ - - 

outputs - Outcomes - Achievements - what is, achieved in relation 
to what was stated in the goals or objectives. 

a. Problems or needs of the community that were identified ar- 
ticulated, analyzed and/or docw>ented. Priorities given. 

b . Alternatives for meeting needs or solving probl^s that were 
„ identified, articulated, documented. Alternatives that were 

selected. 
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c. 



Solutions implemented or toward which action programs or pro- 
jects axe underway - or on which decisions were later made to 
take ho action. Consider things such as: 

- improvements in community facilities or services, local 
^ economy, land use patterns, etc. 

- new patterns and practices eitablished - problem-solving 
mechanisms institutionalized,i 

- policies changed and/or established. 

- extent to which problems have been ameliorated or solved or 
needs met. 

- changes ft-om situation statement in Plan of Work. 

-'judgements of individuals, ^groups, agencies, public offi- 
cials and plaiuiers as to what has been accomplished and the 
value of the accomplishments to the community. 

_ estimate of man-days of volunteer leadership given to the 
community by leaders assisted. 

d. Alterations or adjustments in organizations or institutions: 

- in program or project efforts, 
in composition (make-up). 

- in purpose. 

- in leadership. 

- in viability. 

- inputs of organizations into solving of community problems, 

e. Impact on people in the communitV ^ as judged by the pebple 
of the ccramunity: 

- on individuals. 

on community government. 

- on relations between community government and the' citizens 
of the community. 

. on community sub-systems - cultural or racial groups. 

- on relations between the community and adjoining communi-. 
ties. 

A proper documentation and axrayal of infonaation *° f 

issues should satisfy the need to show the effectiveness or lack of effec 
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tiveness of the techniques we employ to deliver rural development educa- 
tional assistance in the land-grant system. 

One last thought. Whether or not we like it, we axe in an age of 
accountability in public services. We must be accountable for what we say 
we will do and for the public funds expended to do it. We must be account- 
able to those who provide the funds. Or, we must decide that we don t wish 
to participate. That is always one of our alternatives, of course. I would 
prognosticate that those programs in the public sector for which the leaders 
axe reluctant to be accountable will not be discontinued iinmediately. What 
we might expect is a gradual decline in support, beginning with a decrease 
in the rate of increased funding, and a shift of funding to newer pieces of 
legislati<?n which impose controls with which leaders must comply or face 
criminal or civil action or total loss 9f funds. 
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THE EVALUATION OF DEVEWIMENT PROGRAMS 
Edward 0. Moe 
INTRODUCTION 



This workshop on evaluating Title Vprograzns provides t^ 
ity .for a valuable exchange of ideas. The context within ^^^^J ^^/^'^ 
TnJludinc the array of expectations different groups hold for Title V, 

!^^oli Lr^cUxified! Alternative concepts of development and eva 
dti^irbe^xaSiSd M^ch can be gained fro. hearing about the pW 
■«nd ^hHla^s ?revaluation of the various states. Discussion of the 
SLs wiai swst wayc in which they caii be strengthened and through 
plans W14.J. """^^ „ . +- -.-rftotice and knowledge. 



•nlans will sviRgest ways xn wuiuu oncj — ° , 

SSh Sy c^contribul^e more both to practice and knowledge. 

The i^apersby Olaf Larson, Pat Madden and Howard Tanker sley provide 
the overvira^d perspective needed and^^^^^^^^ 
proaches to the tasks we confront. What I have to say wxxj. d.ux 
has' been said. 

Title V seems to make more sense, given the g^andness of the design _ 

city at two levels. 

capacity at the community level to initiate and sustain develop. 
S! to solve community problems,, and to make continuing ad- 
Oustments to the larger environments- 
Capacity on the part of land-grant institutions and other helping 
Sencies ?a provide the assistance- and. resources needed by local 
opJe Ld local institutions to help make their efforts ef fee- 



1. 



2. 



'tive . 



It follows that creation of new capacity at these /^^^^^/^^^Ji!! 
both iltMn ^ch level and between levels new ways of working, new linking 
mecha^lirs" new systems. These are what we are trying to -invent. 

For decades attempts at development in rural and non-metropolit^ 
areas SIve faced a haxd uphill battle. Industrialization, urbanization. 
Sd'uira^cratization of American society resulted B-^^^" ^^3^^ ^^^n 
concentration of population, economic activity, aijd social services in 

■""^^ 0. Moe is Principal Sociologist and Co-ordinator Hural^ 
Development Programs, Cooperative State Research Service, U.S. Department 
of Agriculture, Washington, D. C. 
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high substitution of capital- for labor reduced ^^^^^^g^t^ • 

i t2.v of alternative employment opportunities resulted in population 

d^n^^aiSd a- SneSSJ deSS^ting St of conditions in rural c-un.- 
ties.^ 

These forces and their consec^uences may have "bottomed out" . 'Beale 
(197Urhas scS highly sign^icant observations about "popalation 

and settlement prospects". \ 

the nation appears to haVe arrived at a point vhere a ' . 
triliism of generati<>ns - that Ai^rican metro areas increase 
in population moreXrapidly than^onmetro areas - is no 
longer valid." \ . \ 

"Eniployment data support the population statist.ics. Ftcm' 
March 1970. to March 1973, data derived nationwide from 
State Employment Security office files show an increase 
of 7 8 percent irv nonmetro area jobs and 3 .6 percent in- 
creLe metro areas. Unlike tlie 1960's when manufacturing 
wl^ the only major group with higher growth rates in nonmetro 
TesT, tSelW.1973 figures indic8!te higher nonmet^o^growth 

every comi^nent except government. This means that the 
Irle Zd services sectors of ruxal. and small city -areas have 
been advancing as well as the goods -producing seqtors. 

There is no radical shift. We will continue to be an f ^^:'^*J°^?^^3^^ 

^t^^^i:"^^ ^^^^ 

stand both the forces and their consequences . ^ • 

S if^sible to have greater effect with a given investment, of re- 
' Purees. This brings us, then, to development and evaluation. 

Developnent theory, research, and practice in the United Stat^ and . 
thro^Sout Se worla is in a period of searching reassessment^ So^_ 
tSrtfyears of experience have accumulated since the Planning of *^e 
ZTdL and widS; hailed development I-ograms °f P-^-Wor^J II . 
Despite some^ conspicuous achievements in the United States and in o^n 
co^tries, advancements, in developnent to date hav6 not been ^^fj^^ 
ZTlo^TB Twhether -theoreticians, policy makers, planners^ programmers,. ^ 

are dilappointdng . The reasons are M *° 
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dn, the confusions and complexities inherent in the nature of developnent 
and evaluation. Assumptions about the desirability and appropriateness 
of developaent, and acceptance 6f the American experience a? proof that 
we taiew how to dssign and conduct developnent programs, tewded to play , 
down- the importance of evaluation. At the same time the confusion and 
lack of clarity in general' design; in goals and objectives; in policies^, 
prtjrams and procedures; in measures ox yardsticks for developnent; and 
in the conduct of developnent effbrts made evaluation ^remely difficult. 

Other complications, of course., are inherent in the conception, 
design and processes of evaluation. Evaluation is most effective when it 
is a clearly designed, built-in, continuous part of developnent programs , 
In i^actice, evaluation rarely met these conditions with the consequent 
loss both of solid evaluation and a better understanding .of the riature of 
developnent itself. 

The set of ideas in' this statement is an attempt to identify and 
outline some of the qritical considerations in the evaluation of develop- 
ment programs." From this set of ideas, critical elements can be Identi- 
fied in the design and. evaluation of specific developaent .programs. 

MAJOR SECTIONS OF THE STATEMpT' ABE : 

' / 

I.. CW THE NATURE OF DEVELOBffiNT 
II. A CCaJCEPTUAL VIEW OF DEVELOIMENT AND DEVBLOmE,HT PROCESSES 
III. THE NATUIffi OF EVALUATION: ' ITS FUNCTIONS IN DEVELOIMENT 

IV. SCi?IE GUIDELINES .IN THE EVALUATION OF DEVELOHffiNT 'EROGRAMS 

v.. SOME CRn-.:CAL EIEMENTS IN THE EVALUATICai OF TITIE V 
. ' mCGRAMS ■ , . , ' 
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I. 'OK THE KATUBE OP DBVEWPMEMT 



Searching questions" haVe been raised aboixt what develoi»nent 
is, and what distliiguishe-^s it from something irtiich is not devcl- .. 
opient. There is general recognition that the concept is at the 
s«e tine both deceptively -simple and hcrrendously complicated. 
There seems to.be sc«eirti&t general acceptance>of the idea that 
it connotes Improvements in the position of people and social 
and-econottic entities - organizations, communities, regions, 
nations - irith respect to certain key values such as > " 

wealth, education, health, w^ll^being, dignity^ '^^^^f^^^"-^'" 
cal effectiveness, and freedom. It is «l8o thought to Jijply,^ 
whether used In botany, engineering, psychology, child develop- 
^nt, economics, sociolog^Tor other disciplines . . . an u^olding 
or movement toward . scmeTets of conditions or states of afffrs 
regarded as desirable such as adulthood, maturity, self-susta^ed 
socill and economic growth,. .differentiation, or quality of life. ^ ^ 

One may view the .conceptual difficulty, in part, as rooted 
in the so-called nonnative problem, 9eveloittent ^^""f 
nLwttive concept, and one can not - L with it^withcmt^vokin^ 
both^intem^ a^d external yardstick, in analysic as Warwick 

(19^) observed. 

Social^ scientists, in pakicular, are c*^^*,^?,^ J^*' ^"^^^^ 
tive problems aiid the underlying values^-Hssues. It^has been much 

easieffor social scientists to talk "^-^ ^^^f .oSs "^hS 
. change or modernization. These are more neutral conceits. Th^ 
deS^with What is teippening or what will happen. When one^^-es 
the tena development or underdeveloiment, there is . the WiciJ 
co^t"ion of Someone's idea as to what ^^^gP*^^^^**^ 
^o^^ities and societies'. It. is in this way that botJ^^^«l 
and external yardsticks become involved in choices of what people 
^t!lnd^nhey establish as goals and objectives of developnent 

programs. 

Differentiating Ideas on Development 

A useful question to confroht is the; differentiating ideai-fl 
the concept of development. What distinguishes developnen.t from^a 
art S conditions or Tstate of affairs which is not development? 
The writer had the privilege of exploring this question with some 
f/dty Cornell f a6ulty and. graduate students from a wide range of 
disciplinary and experience backgrounds (Moe, 1971). 

' It was possible to lay out a number of enphases which- charac- 
terize ways of thinking about, development. Emphasis as used in this 
coSe^Xs Lt Wthat a person leaves dut, ignores or does not 
.recogniz^. other aspects and iinplications of the concept of develop- 
ment: .It is significant, also, -that every person interviewed saw 
deveioSent a^lssentially interdisciplinary or requiring^sophis- 
ticated interdisciplinary contributions to build on and -extend the 
perspective of the separate disciplines. 
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The major emphases abound vhich the various point, of view can be 
described are: 



Goal-directed, goal-oriented change 
Preferred states or levels of well-being 
Use of rejsources 
Capacity 



of individuals 

N 

Capacity of systeas 

Increased levels of productivity 

Quality of life 

Sets of conditions characterizing develoi«nent . . 

Level ^'6f differentiation 

Number of options . - 

Surplus of product/ skili. 
Each of these einphaaes vas briefly exandned in the paper referred to above. 

The Confusion Abomt Develogaent 

It has been evident for. some time that great confusion character- 
ized' ^L^^htSS conceptualizations of development, f ^^^^^^^^^^ 
more . Sent as the experience of last quarter of a cetit^^^^ 
been revieved and re-analyzed. In this section of the PaP«r» the 
m^^asi^cts of the confusion as seen and «JE>«^i«°°«^tX JSSiration. 
^Tcultylfadministrators, and graduate students i«. J ^ ye didn'i 

A nu»bei-of faculty members put their view succinctly: • ; ' ^ f ^ 
knoW vhat it was, but we thought it was- a good thing. We are about in 
the «aae situation now." 

The major confusions expressed in the interviews are these: 

lick of parity about what develojsaent is. 

Failure to deel constructively with the valu4 problem. 

Overdependence on aggregated quantitative and/or narrow 
^ quantitative ideas and measures. 

Equating means with ends - technology, structure, , - 
institutional fonrs, process. 

Fragmented and singlfe disciplinary approaches. 
Moralistic speculation. 
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External manipulation. - high profile of the helper, 
helping agency or helping nation. 

Failure to anticipate and project consequences, and to 
see second generation problems. " \ 

i 

Endemic Problem s in Planning and Developpient Processes 

— ' ^ 1 ^ 

Some significant assumptions, both ideological and managerial, 
underlie current thinking about planning and develo^pment . Among 
these are: f 

1. So»e re-emphasis ou the role and responsibility of local 
government, local public and private^plaiming and devel- 
opwent agencies and local leadership) within the Federal 

.system, / 

Z. Soae ideological reassertion tha^t local government is 
best able to identify and solye' local problems and 
initiate and direct developeient programs, and 

3. Reconsideration of , questions of scale with emphasis on 
decentralization. 

Continuing analysis of the planning and development programs 
of the sixties and seventies points mq) some basic and persistent 
problons. These seem to have compromised efforts to achieve the 
'igreat society", and to build system capacity at various levels. 
A National Acadawy of Sciences, National Acadeiny of Engineering 
report (197V) re-identifies these basic considerations: 

1^ Lack of congruency between locally perceived needs and 
national priorities and programs^ 

2. Failure to match statments of priority with resources, 
personnel, and funds. 

3. Tendency for problems to be greater or smaller than the 
jurisdiction of the relevant unit of government. 

Failure in planning and development to encompass the 
full social, economic and political dimensions of 
probl^ems . 

5. Failure to integrate planning and development with 
political processes of decision making and allocation of 
resou3j»ces. 

6. Inadequacy of research and evaluation tools (both the 
state of the arts and the use of ^at is available) to 
essess processes and ascertain impact of programs; 
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In one way or tinother, we continually encounter these problems in 
developing, implementing and evaluating programs under Title V. 
SS^^^ectlves xaaybe different, funding levels may vary widely,. - 
but these kinds of problems axe always there. 

II. COHCBPIUAl\iEW of DEVELOBffiKT AF JEVEIX)PMENT 

PROCESSES 

A simple framftwork for thinking abou> -^velopment and for 
analyzing development programs has emer^.a out of past f ^ pref 
work. Development programs or efforts may. be analyzed in terns of.. 

1. Clarification of the goals in development, and what de- 

, velopaent is defined to be - in specific terms. What is 
supposed 'to occur? What axe the ends or the desired 
states to be attained?^ 

• 

2. Identification of the u^its within which development is 

to occur. Are the changes desired changes in individuals, 
organizations, communities, multi-county units or some otner 
units, or are changes to be made concurrently in several- 
units and in the relationships among units? 

- 3. Specification of the processes and strategies by which deve^^ 
opment i.s to be attained. These processes and strategies need 
tr be defined in very specific terms and examined as to their 
api««opriateness. for dttaining the ends specified. . 

k. Internal and external measures by wl|.ich development is to be 
gauged or evaluated . What are the ci^iteria by which some 
determination can be made as to whether development has 
occurred or the extent to which it ha\ occurred? Again, 
great clarity and specificity axe re^[uired. 

Developtent, while difficult to attain,' is «""}atnguing idea^in 
human socie?^. Ufe and events do not-have a ^^^^^^^if !f o- 
from past to preuent to future. We have been successful in making^pro 
Sion^ of what conditions will be like five, ten and fifteen years 
So the future. Building on such Projections,. develoi«ent^pres^^^^^^^ 
the opportunity to go beyond projections to more Pf«°i»« J«f ^Jj^J^^e 
of what we would like to see exist, or what the alternatives ^if J* be 
?f S^s^f^e desired future states. If we are clear «^P>jt^the^ends , 
it is likely we can devise the policies and the "^^'^tegies to achieve 
the ends. It least we will know more clearly what we are about, and the 
nagnitude of the task we confront in developnent . To the extent that 
Jiedo this, we cnn come into the present from the future as well as ^ 
from the past. 

A Chain of Events 

Developaent connotes movement from what exists to the achievement 
of ends defined as the goals of d'svelopnent . The ends may be certain 
desirS states, ^rtain^ondl =-ons, or the initiation of new procedures 
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and processes. .The movement is considered to occur 
NOT a chain of ^Slh^d events.. These planned events are initiated as 
bei^ aS?<^ri?te, or the most appropriate, vays of achieving the ends. 
A nSbefof attem^As have been made over the years to define the se- . 
- n„P^ of ateus The major "schools" in community development have 
Topo ed "Si;- IfZis and describing the steps. Son« examples are 
the models used at Michigan State, Missouri, Southern Illinois, the 
S^ive^ity S Chicago, the University of Washington, the Universitjf^ of 
ma^ "ieVi^deSiaSl^^el; and other s^^(Moe, 1965 . ^/,°r^^,^\^^^^ 
ment models are somewhat similar to "social action models such as Beai 
Tl^)or to the stages of policy development suggested by.^Teeman (1970) 

One view of the major phases or stages in ^^^^^^^^^^.J.^ "^"^^^333 
below. Some of the steps within each phase are also identified. These 
ISZs and the steps within them ^e always °n-B°i"f. i^*^^^^^*^^' 
never as discrete and separate as they appear in written form. ^ 
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II. 



Pluming Mechanism' Adapted or Created 
Mechanisms for Citizen Involvement Devised/Used 
Problem/Situation Analyses; Needs/Wants 

Assessments Made ' • . - 

Search for/Analysis of Alternatives •» 
Goal Setting/Policy Formulation Initiated 
Monitoring/Assessment/Evaluation Initiated 

QRGANIZING/MOBILIZm BESOURCE S FOR- DEVELOBffiMT 

Continuing Planning/Further Speci^cation of Goals 

and Objectives' 
Continuing Citizen Involveinent 
Designing Education/Action Programs 
• Adapting/Creating Organizational Structure/Procedures 
Identifying/Developnent of Leadership 
Identifying/Gaining Access to Local/Outside (or 

Internal/External) EesoiK'ces 
Monitoring/Assessment/Bvaluation 

III. CONDUCl/lMPI£MEmTION OF PROGRAM 

Planned Activities /Events Conducted ^ 
Continuing Citizen Involvement" 

Moni-toring/Assessm8nt/Evalua|;ion 4..„„„t o+^„o+nrp/ 

Modification of Activities/Programs/Organizational otructure/ 

Procedures , ' , 

IV. BEPIAMIMVRECYCLING-OF PROCESSES 

Goals, Programs, 'Activities 

There are 'many different ways in which developnent may be pursued 
and the sequence of events described. This is one ' "J^^fg^w 
way, nor is it intended to be a definitive statement. It i^^^fficie^ 
Slete however, to use as an illustration and as a point of reic.cnce 
f^cSrl^an approach to the\valuation of development vro&:^- It 
vSl enSlTSaminaUon of inputs:, outputs, processes and products at 
vca:*ious stages of a development PQ^^fG 
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THE NATURE OF EVALUATION: ITS RINCTIOHS IN DEVEIX)PMENT PROGRAMS 



Most definitions of evaluation in coHsoon usage identify two 
significant aspects of the process • (l> the deteraination or measure 
of "the quantity of something such as activities, programs, events,, 
inputs ,^tputs, impact, and (2) some judgment of,^or determination 
offthe value or worth of what has been done or what . ^ 

This is frequently expUssed as a kind of general measure of the costs 
and benefits and 'seme balancing of the benefits against the inputs 
or costs. This should not be ,confused with more specialized cost- 
benefit analysis, which is a particular form of evaluation. 

As evaluation is used in various public and private organizations 
and agencies it tends to take these forms: f 



(1) Monitoring 



• the atteiapt to determine whether 
agreements or contracts or pro- 
grams are Implemented according 
to an accepted plan and the 
attempt to determine how they 
might be l»ple»ented more effec- 
tively. This is closely related 
to procQSSUal evaluation described 
below. 



(2) Performance Evaluation • 



(3) Processual Evaluation 



Program Effects/Outcome 
Evaluation 



(5) Impact Evaluation 



the measurement and interpretation 
of the performance, of people in 
various positions in designing and 
implementing activities and programs, 

the building in of an information 
and feedback system to provide data 
for modification and improvement 
of xtt^ograms, processes and proce- 
dures while they are being implem- 
ented* 



the attempt to determine the extent " 
to which> and the effectiveness 
with which, program objectives 
are achieved. 

a broader concept which involves 
not only the outcomes but the over- 
all Impact of the achievement of 
outcomes on the total community or 
the total system, and on individuals,, 
groups or significant subparts of 
the system. It is a form of evalua- 
tion that, attanpts to get at the 
intended and unintended consequences 
of policies and programs-. The em- 
phasis is on an estimate of what 
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happens and not on the pre- 
cise determination of specific 
prograja efforts. 
< : ■ ° 

These are significant interrelated aspects or emphases In ^ 
evaluation. In one iray or another, they are all expressed in the 
more consistent and continuous attempts to plan programs and de- 
termine their outcomes. These aspects of evaluation take on par- 
ticular significance in assessing development programs. 

A Conceptual View of Evaluation 

To begin with, evaluation is a fundamental process in all 
human activity. Looois, for example, identified evaluation m a 
basic process in his model of a social system. Normative activities 
8uch as developoent occur within a definable, context or set of cir- 
cumstances. People in conmunities, multi-county regions or states - 
or nations, for-example, enjoy a way of life and an array of goods 
and services. They also experience problems, and have wants and 
hopes and expectations that are unfulfilled. Income may be low ^ 
according to both internal and external yardsticks . Unemployment My, 
be videspread. Significant ifesourceS may be present whi^ could be. 
developed to expand the empldynent base and the local economy. 
-Health and.medical care may 'l^e inadeqfuate. Career education 
programs may be needed in thi . schools to help young people plan^ 
theS" -futures around their oU educational and occupational aspir- 
ations and expectations. Tile capacity to initiate and^ sustain . 
change or development or maie needed and desired adjustments is _ 
inademiate. A development /process such as outlined above i« ini- 
tiated. Seme planning is >rie. Goals and objectives are estab- 
lished. A development program with a^- supporting plan and develop- 
ment organization is launched. Action and educational programs 
are designed and implemented. 

Evaluation of development or other programs or activities may 
be conceived as having these major aspects: 

(1) Defining the context within whiclJ development 
is to be pursued , including establishing some 
base lines or known starting poijits from which 
planned attaapts to achieve goals and objectives 
can be measured. 

(2) Documenting the inputs, the things that are 
done, the programs and activities initiated 
and the resources used to bring about dmprove- 
aent and to achieve the goals and objectives 
that have been set. 

(3) Documenting the processes or the ways in which 
programs are Implemented. It would detail ways 
in which planning is done, decisione made, com- 
munication channels established, the interaction • » 
patterns that emerge, the critical incidents that 
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occur, the interpretations of and changes in policies, 
I and other such feattires. 

(U) 'Documenting theXdutputs or the outcomes or effects of 
vhat is done • Tp^what extent are the goals and objec- 
,x tives achieyed? Questions about the effectiveness of 
the activities planned in achieving the goals are also 
to bJi considered. 

(5) Assessing the overall impact of the program as a whole 
and its major activi^fies on, the total system and impor- 
tant subparts of the syston such as different groups of 
people and different political units. This is a signi- 
ficant overview which goes beyond the more specific 
questions of the ways, and the extent to which goals and 
objectives are achieved. The concern is basically with 
consequences 9 both the intended .axid the unintended. 

The Basic Functions of Evaluation 

Evaluation including evaluative research is viewed as having 
five bfiisic functions: 

(l) Enables the selection of goals and action and education 
ea^iernatiyes in achieving goals which eo'e most likely to 
be successful* 

This sTiggests that evaluation begins irtien djevelopment begins • 
The measixring- judging process is distorted yftAn it is seen 
only .in terms of the achievement of program goals. 

Builds on the collection of data and feedback heeded for 
correction aAd adjustment while programs are underway, and 
enables keeping activities in line with goals.. It also 
provides data for the modification of goals* 

Enables determination of the extent to which , and the 
efficiency with which, goals have been achieved. 

Enables awtking some overall^ assessment of the impact of 
programs on people, on systems and systdn processes, and 

Enables docrmentaliion and use of Artiat is learned about 
designing and implementing deyelopment programs. 

In one way or another, something is done in relation to all of 
the functions whither or not there is an e3q?licit evaluation effort, 
^formation is pulled together. Alternatives are defined, judgements 
are ma^e, and action taken* The^,process may be unsystematic, piece- 
meal jMid fragmented but it occui'S. An explicit evaluation effo^t*^ 
attempts to make the process systematic and to assure effective 
pursuit of the functions* 



(2) 
(3) 

(«*) 

(5) 
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Inmits and Outputs In The Develoimen t Process 

As pointed out above, evaluation is considered to imve five 
iBa:5or aspects - definition of the context and establishnent of 
, soie basflines or known starting points froa which measurements 
and judgements can be 'made, a documentation of inputs, a docil^ 
mentation of processes, k documentation of outputs and an ajs^s 
ment of the overall ixpact on the system or systems and their \ 
significant subparts. Before discussing the problems of inputs, 
processes, outputs, soiae clarification of the context and base, 
lines or known starting points is needed. 

Expenditure of effort in a program more or less assumes 
that something will occur, and that the choages will be in the 
■ direction defined in the\oais and objectives. Determining 
whether or not the changes planned actually occur requires s^e 
specification of what the initial conditions are. The specifi- 
cation of the context and .number and kinds of base lines needed, 
depends, of course, on the nature of the program. There has been 
some tendency in research to spend more tUne and effort on con- 
'texts and base lines, and to collect more data than is heeded. 
The definition of contexts and base lines should be a^^^*** 
not overly elaborate. In an early evaluative study of the diffu- 
sion on agricultural technology, the base line was ^- 
adoption of 12 recowncnded practices by a small carefully selected 
sample of farmers. It proved adequate. 

An ingenious example of a base line emerged in a consultative 
session on the evaluation of the work of a newly formed area agency 
on aging in a multi-county area in the Southwest, lUe °^ 
the^ea Agency is to mobilize resources and encourage the design 
and modification of coBmhmity services to enable elde^y P«opl«^J° 
continue to live independently in their own homes. After ejP^oring^ 
many possibilities which were far beyond the resources of the^agency. 
It "was proposed that the current ra^e of admission of people _to 

nursing'^hoS^s in the area be accepted }>'^f'^V^,J^,^'^^^Ze 
of the rate of admission and the types of l^°Plf .admitted would^^ 
possible three significant kinds of findings, (l) did building in 
new services for the aging affect the rate of admission? (2) had 
people been admitted previously who could now return to their oim 
hSes? and (3) what kinds of services were needed that would enable 
others being admitted to continue living independently? 

Base lines or known starting points are needed. They need not 
be elabo7. -ie and costly. The more thought given to their selection 
and use, the simpler and more appropriate they toe likely to be. 
We can now consider inputs and outputs in the various phases of the 
develoiment ^process . 

In the charts that follow, an attempt has been made each 
phase of the developdient process outlined earlier, to identify In-^ 
puts and outputs and products appropriate to that phas?. The inputs- 
outputs identified are illustrative of those that might characterize 
a specific develojmcnt program. 
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Need for Stratef^lcal Thinking said Stra tet^ical Planning 

involvement In development programs accentuates need for strate- 
eical thinking. If we can conceive of developnent as occurring in stages 
overtime aS? if we can plan and do within each stage what is critical to 
IIZ JZe ^ iill eSiL our chances of success. Development goaJ.s in 
t^f eSS stl^ef of programs are somewhat unreal.. They acquire significant 
:ranLTLl?~Vhem, and move toward the, ^hg^^ "^tf ft^es 
with in this workshop is a way to bring this about. Clariping xne 
Sid bimS, in evaluation as a self-correcting mechanism is a long step in 
the right direction. 

■IV - SOl^ GUIDEHNES IN -THE EVALUATION OF DEVEK)PMEm PROGRAMS 

I 

Reflection on some thirty years of experience and qbseryations of the 
field of evaluation leads one to the conclusion that some principles or 
Sidelines Tave been identified. They represent a useful summary a^d, as 
such, are presented her.e. ^ 

1. Evaluation is more effective when it is designed within^the hier- 
archy of the explicit and implicit ends and means operating to 
Side programs and within the specific sequence of activities 
planned to achieve the ends. 

Development moves through a chain of means and ends in which the 

Sevrent of many ends or goals f ^°^J-*^-/,^,re°i: St 
achieving other ends. Planning and plans are a °f ;. „ 
While the 'building in of planning as a process and the completion 
S iri^Jen pla^slre. signif icant ends, they are perhaps taore Impor- 
t^t as means to other ends. 

Evaluation to be effective mus.t be designed within ^^^^^J^^^^^^f 
means and ends. It must also take into account the strategies ana 
suSieUes involved in the setting 6f goals and objectives and the 
devising of means. The more conversant evaluators are with a de- 
velo^St agency and with the\ factors which affect its progrms, 
inclJ^ing its concern with survival, funding and relationships to 
other agencies, the more pertinent and realistic evaluation will 
be. , 

2. Evaluation will be more effective when it is an integral, built-in, 
continuous part of a development program. 

When evaluation is an integral, buUt-in °°"tinuing part of^a de- 
vplorment OTOcram, it can best meet the conditions identified in J. 
IbovrVSsHnables building in of an infonnation-feedback sys- 

tem that strengthens every phase of the ^^^^ ^X^^tives 
helTJs in the selection of more promising development alternatives 
Sd the means for their achievement; it helps keep the f °f am and 
StivitieHeared to the goals; it helps modify and re-define the 
«oalI when necessary; enables the detenninatioh of outcomes and im 
Ind iJ^^rt^y; it helps document what has been learned and 
e^lAdrthftoowledge and experience base on which new development 
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programs may dray., ' ; 

'Evaluation is more effective when it utilizes the knowledge and 
experience of program designers, field staff, researchers, and ad- 
rainistrators, and identifies .and serves their special needs. 

There are significant differences in the needs and interests of 
different individuals and groups in the evaluation of any Progra^ " 
or activity. To -,he extent that an evaluation design can identify 
and respond to these needs, as veil as those which dea^ with pro- 
gram effects and impact, the more effective it will likely be. 

Evaluation will be more effective when it explores in the build- 
ing of a design or a plan for evaluation the critical issues and 
^potential conflicts about goals such as the agency's goals, the 
^community's goals, and the ^ggals of special interests; when the _ 
implications of negative outcomes are explicitly examined; and when 
a substantial- consensus is established on such critical issues. 

DeveWent programs usually involve a wide array of people, 
groupsVpublie and private organizations and agencies, and communi- 
ties aAd political units. Title V is an excellent example. Their 
goals and objectives as well as their reasons for participating in 
a development effort are likely to be incompatiole and may be con- 
flicting. The genius in designing development programs is to in- 
crease compatib^ity in goals and e^ectations and to reduce the 
likelihood of conflict.. Evaluation can be an effective tool both 
in understanding -potential conflicts and in continually building 
in ways of 'strengthening consensus at points where it is most 
threatened. 

Evaluation is essentially ah internal process. It is most effec. 
tive vhen it is designed and' managed by an agency to assess its 
own program and activities, and when it has built-in safeguards to 
protect the agency against its biases and its myopic view of real- 
ty- 

Evaluation, while it may be done by an organization itself or 
done by an oxit side group, is essentially an internal process. The 
greatest potential gains to an agency are those related to in- 
creased knowledge and skill about what it is trying to do . For an 
agency concerned with development, it is the knowledge and skill 
about how to- design and implement development programs. External 
observations, studies, and audits have a significant role to play 
in .protecting an agency from its own biases. The uses of /"dh de- 
vices are to be encouraged, but they should supplement and extend 
rather than confuse the internal management of and responsibility 
' for the evaluation process. 

. Evaluation is more effective when it utilizes the knowledge and 
competence of research and evaluation specialist s.in the design of 
evaluative studies and evaluative criteria. 

As indicated, above, effective evaluation requires maximum use of 
ideas, knowledge, and skills of the staff members of an agency. 
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This may include special .knowledge and competence in evaluation. 
When it does not,- such technical assistance should be obtained 
from other sources. Effectiveness is further enhanced when such 
•"outside" specialists have a good understanding of the type of 
program being evaluated. 

Evaluation is more effective when all the types of people and 
agencies participating in a program, or affected by it, axe in- 
volved in the attempt to determine and interpret outcomes and 
' impact , and what is' learned . 

Involving people and agencies who participate in or who axe 
affected by programs in their evaluation assures a more accurate 
and comprehensive view of y/hat has been done. Such groups have 
information and perspectives that are critically needed. Fail- 
ure to involve them could lead to incomplete or erroneous inter- 
pretations . 

8. Evaluation and developnent programs themselves are more effective 
when these conditions are met: 

a. Some careful realistic^descripbion and analysis of 
context is made and carefully designed base lines 
are established fr<»n which potential changes can be 
measured and interpreted. 

b . Some simple ways of monitoring activities and pro- 
viding \iseful feedback axe determined and built-in. 

c. Some significant criteria and measures of effect 
and impact are defined, and 



a. 



When the responsibility for evaluation is assigned 
and understood. 



The conditions axe a further specification of the guidelines stated 
above and the discussion in the paper itself. Establishing these 
conditions means that a design has been created and enhances the- , 
likelihood that it will be used. 

q Evaluation is more effective when the initial acceptance of a 
workable plan is strengthened by competent, consistent implemen- 
tation of the plan. 

The real value of an evaluation plan cqmss into focus when it is 
implemented with imagination and understanding, and the kinds o^ 
data needed to make decisions are actually provided. Failure to 
provide such data for whatever reason - a poor plan or inadequate 
implementation - threatens not only the evaluation effort but the 
developnent program itself. , 

10. Evaluation is more effective when an attitude ^^"J^^J^"^ 
and mutual trust is established in the design and implementation 
of an evaluation plan. 
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Attitudes of non-defensiveness and mutual trust are critical in 
evaluation. When they exist, critical issues can be confronted . 
and reasonable solutions devised. When they do not, attempts at 
^ evaluation become painful and axe avoided. Development efforts 

are Impeded and learning frustrated. The choices are never easy. 

These guidelines are one way of summarizing a perspective on evalua- 
tion. Based as they are on observations and reflections over a consiaera- 
ble period of time, they represent an attempt to sor\ out what seems to 
have worked. The probabilities are high that evaluation components of de- 
velopment programs which meet these guidelines will be more effective. 

V. SOME CRITICAL ELEMENTS IN THE EVALUATION OF^ TITLE ^ fROGRAMS 

Title V has been interpreted as an attempt to assess the capability of 
land-grant institutions to deliver research and extension ^assistance in de- 
velopment. This idea is implicit in the pilot nature of the program. As 
is so frequently the case, however, the major .political decisions will have 
been made before even one year of the three-year pilot effort is^comgeted. 
Those who looked realistically at the time table were aware of- this from 
the outset. Funds became available December 1973 and State Plans of 
Work were due in Assistant Secretary Ix)ng«s office February U, 197U. Some 
Plans of Work ^rere approved before the due date. The last program to be 
approved was dated June 27, 1971+. At this writing, early consideration of 
the '77 budget is underway and by February 1975 ;Will be well along in the 
process. Its general form will have been -set and most major decisions, at 
least at the departmental level, will already have been made. At best only 
one year of a three-year pilot effort could be taken into account. 

It is not in keeping with the facts of the situation then, to design 
the evaluation of Title V to influence political decisions on Continuation 
of the program. We are forced both to a:more solid and basic justifica- 
tion - and that is to learn as- much as we can from the attqnpt to deliver 
develoiment assistance through a combined research and extension f°sr^' 
We can sharpen our theories, concepts, practice i and evaluation of develop- 
ment. We have the opportunity in some cases to o^^^^f j^^* ^^J5^"^,^^f 
we involve other public and private institutions, including institutions of 
higher education. To do this, of course, we must have good^ designs^for 
evaluation. ' It is these designs and the critical elements in then which 
are of specific concern to us now. 

The basic plan for the evaluation of Title V State Programs, jointly 
developed by the Cooperative State Research Service and the Extension Ser- 
vice, includes these four components: 

1 A plan for evaluation is a built-in part of each State Plan of 
Work rCRegulationa 23.6(a)(6))l. Emphasis is placed on assess- 
ing outcomes in relation to ob3ectives, the effectiveness of 
extension and research program techniques, and the organizational 
structure for planning and conducting programs. Appraisals by 
community leaders in the areas in which programs are conducted 
will provide significant data. The Cooperative- State Research 
Service and Extension Service will analyze these reports and 
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prepare summary statements. 

2. The Cooperative State Research Service and Extension Service in 
three ai%&s - economic development, Improvement of community 
services and land use - will work with the States in doing a more- 
intensive evaluation. Information from the State evaluation and 
some additional information will be used to summarize .inputs and 
outcomes and prepare summaries and generalizations about effec- 
tiveness and impact. 

3. Four states - Georgia, Kansas, Rhode Island and Utah - one in each 
of the four regions, have been selected to do an intensive docu- 
mentation and evaluation of their programs. These studies will., 
identify changes from a baseline and supplement facts and figures 
with pictorial (audio-visual tape) documentation. 

U. Using the three components above - the States; own evaluation, 
the assessment by. type of program, and the in-depth and film doc- 
umentation - CSRS and ES will prepare a summary document on the 
overall impact of Title V pilot programs. Particular attention 
will be. p^d to such factors as cooperation between Research and 
Extension, the involvement of other institutions of higher educa- 
tion, and the strengthening of the system for delivering research 
and development assistance. 

Some" Critical Elements. 

While it might be too much to expect, one can still hope that Title V 
evaluation plans will conform in significant respects to the guidelines 
ouUined above in this paper. One could hope also that the basic functions 
of evaluation as described on page k2 would be met in ways^which m old 
strengthen the program, enable us to interpret outcomes and impact, and to 
document what will have been leax'ned. 

If this is to happen, there are some specific elements or considera- 
tions we will need to take into account. These are the major points around 
which documentation should be planned. These are outlined below:. 

1. Progress Toward Achieving the Objecti ves Stated in the Annual 
Plans of Work. 



a. 



b. 



c. 



d. 



Problems identified and addressed in the Plan of Work. 

Stated goals and objectives - changes in the initial 
statements of goals and objectives to be documented, 
including the reasons for modification or adding new 
goals or establishing different goals. 

Statement of the basic strategy to be employed in the 
program - how are changes to be brought about. 

Establishment of some baselines - a specification of 
some known starting points from which measurements and 
judgements can be made. 
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Inputs - a description of what was planned ^d what was 
actually done to achieve the objectives, "botl^ those 
identified in b above and those that are intermediate 
in the means-end hierarchy to achievement of the stated 
goals and objectives. 

Inpucfcg vould Include ; 

(1) Planning - plans and^ specification of activities • 
to "be conducted. 

(2) Activities conducted, including educational, 
'infonnational and. action programs and events. 

(3) Organizing and the strenglihening of old organi- 
zations, and creation of new ones. 

(k) Processes initiated/fffcrengthened - new patterns 
in planning, problem-solving and decision-making. 

(5) Involvement of people^ organizations, agencies, 
potential users of research findings or other 
information, incltiding creation, of new mechanisms 
for involvement. 

(6) leadership recnaitnent, developnent, training. 

(7) Technical assistance and consultation- provided 
or secured frcin other agencies. 

(8) Access gained/use of new financia3./technical 
resources, " / 

Oufcccmes - what is achieved in relation to what was stated 
in the goals and objectives, 

(1) Statements of cccmiunitir/county/ai-ea problems and 
needs written- and accepted by significant groups/ 
citizens. 

(2) Developnent alternatives defined and documented and 
feasibility determined. Develoiwent alternative se- 
lected. Plans made and implemented. 

• (3) Problems solved or ameliorated . Changes from base- 
line conditions. \ 

- improvements in community services and facilities; 
income and wealth; the^local economy; use of rer 
sources - land, water - e;i;c.; 

- new patterns and practices established - planning, 
problem-solving mechanipros instituti'onaUzed; 
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- policies changed and/or established in line with 
program goals and objectives; 

- nev processes established - such as planning 

being accepted and plans used in making decisions. . 

« new community involveinent procedures ectaolished, 
leaders recruited/trained. 

^ judgements of individuals, groups, agencies, 
public officials, planners as to vhat has ^een 
accomplished and the value of the accomplishments 
to the commmiity. 

Alterations or adjustments in organizations or 
institutions : 

- in program or project efforts; ^ ^ 

- in composition (make-up); 

- in purpose; 

- in leadership; 

- in viability, the ability of the organization 
and the conmunity to solve their ofwn problems; 

- inputs of organizations into solving of coaamunity 
problems; 

- in relationships or linteges between and among 
organizations . 

Impact on people in the conmunity - as judged by the 
people of the community: 

- on individuals; 

^ on local government; 

- .on relations between^ local government and the 
citizens, of the community; , 

- on community syt^ucms sub-systems; 

- on minority groups or groups frequently left out - 
blacks, Spanish-speaking, Indians, the agiJig, 
women, youth , etc . ; 

- on relationships between the camnunity and ad- 
joining communities - effects on emerging larger 
ccomunities, new linldng/integrating mechanisms 
established. " 
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2. Title V Regulations, OrRanizatl onal Requirements 

The usefulness and effectiveness of the Regulations and the organi- 
zational requirements such as the State Plan of Work, the State 
Rural Developnent Advisory Conmittee, the role of the Title V 
coordinators. 14a jor changes or modifications needed. 

3. Involvement of Other Colleges and Universities . . 

The kind and extent of involvement ; the amount of Title V or other 
funds spent to defray costs of such involvement, if appropriate; 
the contribution of other institutions to defining and/or solving 
ccranunity problems; the impact of such efforts on the community., 
if such information is available at the.-'time. 

k. Involvement of Local Units of Government and Regional Planning 
and Develggment Bodies - 

The kind and extent of their involvement; the relationships between 
research and extension personnel and such units of government; the 
role of local and regional bodies in Title V efforts . 

5 Relationsh ips of Title V Efforts to Other Rural Developnent 
Aciiiviiies oi USM and Other Federal or State Agencies -^ 

Title V program's utilization of resources and assistance from 
other agencies;, other agency's utilization-of Title V assistance. 

6 . Relationships Between Extension and Research - 

Identifying and/or addressing the problems olf the ccomunity / Way s 
in which they have worked, on what problems and with what effect? 
Nev; patterns of relationship/new procedures which look promising . 

These arp critical elements. There, are others, but these seem to relate 
to all of- the programs. Each state is to build its own evaluation design on 
its own program taking ints account these elements. 



A FINAL WORD 



The Rural Development Act of 1972 is potentially a f . 
lation. It provides a framework within which local People and local, state 
and national agencies and governments can combine their efforts to both in- 
"in.com?unity and larger system capacity initiate an^susta^^^ 
ment and to facilitate social and economic adjustment. High frustration, be- 
cause 'intentions were not backed by resources, is and will continue to be a ^ 
significant part of the context in which we have to work. Sharpness in the 
deSgn of deTlopaent programs and sharpness in the plans for evaluation will 
helT) build knowledge and strengthen developnent practice. While what we learn 
may not influence policy and legislation in the short term, it may do so in 
•the longer term and fl-cm a more solid base . 
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APPENDIX A 



Summary of State Title V Program Approaches 
Abstracted From Plans of Work for Fiscal Year 197^: Northeastern States 



State 



Approach or 
Substantive Area 



Geoglraphic Area Duration 



Delaware "Gyass roots" approach 

-» 

Massachusetts . " 

New Hampshire " 

New York . , * " 
Pennsylvania ' " 



Start 1st year ^rith 3 years 
one community, 1^^'er 
add 3 from other T/ypes 

Rural towns in fou7 3 years 
contiguous counties 

->83.ect one community 3 yeai^s 
after initial work with 
four 'representing dif- 
ferent t^^rpes 



ew Jersey Land-use control (transfer 
of development rights) 



Rhode Island Land use (information and 
education re open space 
and shore regions use and 
policies) 

Maryland Economic development (econ- 
omic development strategy 
including forming an econ- 
omic development zone) 

Vermont Economic devel6pment (job 

and 'income development with 
emphasis on arts and crafts 
as a business) and human 
resource development 
(training) 

Connecticut Community services (p^sonal 
health services) 



One co^lIlty 

3 contiguous coun- 
ties 



One municipality 
first year 

State 



Part of 1 county 



3 years 
3 years 



1 yeor for 
this part 

3 years 



3 years 



Initially 3 counties 3 years 



The 10 townships of a - 3 years 
planning region 
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state 



Approach or 
Substantive Area 



Geographic Area 



West Virginia Coimnunity services (rur- Fire protection- x 
al fire protection, rur- county 
al transportation, emer- Transport- 1 county 
gency meriical service) - Medical services - 

severa.1 cotmties 



"^Maine^ Rural housing 2 to 5 towns with- 

in 12«town area which 
I ^ ^ , includes parts of 2 

' . countjies 
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' APEENDpC B 

WORKSHOP ON EVALUATING STATE TITIE V PILCT PROGRAMS 
IN THE NORTHEAST 

October 29-31, 1971^ 

ILR Conference Center at Cornell University 
Ithaca, New York 

Sponsored by Northeast Regional Center for Rural Developnent 

AGENDA 

Tuesday, October 29 

9:00 a^m. Chairperson, Olaf Larson, Director, Northeast Regional 
Center for Rural Development 

Welcome - Noland o^/ VanDemark, Director, Cornell University 
Agriculturaix Experiment Station 'and Chairman, NERCRD 
Advisory Committee (Board of Erectors) 

Introductions 

J Local arrangements and workshop procedures 

Chairperson, Lee M. Day, Director De'signate, NERdRD 

Why Evaluation of the State Title V Tilot •Programs: An Overview 
frctn Different Perspectives - Olaf F. "Larson 

Discussion 

Alternative Strategies for Specific Evaluation Purposes - J. I^trick 

Madd^, Department of Agricultural Economics and Rural Sociology, 
Pennsylvania State University 

Discussion ^ 

12:00 Lunch • . ^ " 

2-00 Evaluation, Needs Under Title V'of the Rural Developnent Act of 
1972 - Howard C. Tankersley, Extension Service, USDA 

' Assignment 'for work in small groups - Edward 0. Moe, CSRS and Howard 
. C. Tankersley, ES* - ' 

Small group sessions ^ Discussion leaders ^ 
Edward 0. Moe, Cooperative State Research Service, USDA 
Helen Y. Nelson, College of Humaa Ecology, Cornell University • 
Howard C. Tanker sley. Extension Service, USDA 
* Joan S, Thomson, Pennsylvania State University 
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Wednesday y October 30 

9:00 a.m. Chairperson, Joan S. Thomson, Pennsylvania State University 
Sharing session on small group discussions 

'Critical Elements in an Evaluation Design for Title V - Edward 0. 
Moe, Cooperative State Research Service, USQA 

Discussion followed by meetings in smaLler groups with the J 
content and procedure subject to the needs expressed by the ^ 
workBhop participants 

Reconvene for general discussion with emphasis on how the North- 
• east Regional Center for Rural Development could facilitate 
state evaluation efforts 



Thursday y October 31 . . 

9:00 a.m. Chairperson,. Olaf F. Larson ^ 

Continuation of consultation, sharing, review, critique, and' 
related activities as desired by workshop participants 

Presentation and discussion of evaluation plans of two states - 
Maine and Pennsylvania " 

12 : 00 Add oummerit 
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APPENDIX C 



List of Workshop Participants 



ERIC 



CONMECTICUT 

George E. Whitham, Assoc.iate Director 
Cooperative Extension Service 
.University of Connecticut 
Storrs, CT 

DEIAWARE 

Daniel S. Kuennen, Area Agent, 
Ccxammity Resource, Develojanent ^ 
Ketit and Sussex Counties 
Cooperative vExtension Service 
University of DelaWiare 
Ddvery DE ' — ^ 

Robert L-, Meinen ^ 
Sussex County office 
Cooperative Extension Searvice ' 
University of , Delaware 
Georgetown, DE 

MAJHB « 

Forest M* French 
Camminity Develojment Specialist 
Departoent of Agricultural and 

Resource Economics 
University of Maine 
Orono, ME 

Nancy Melniker> Graduate Research Assistant 
Iinpro.ying Rural Homes Project 
University of-Maine ^ 
Orono, ME » « 

MARYIAND * 

Keith J. Guerin 
Extension agent 
Cooperative Extension Service 
University of Maryland 
Cambridge, MD 

Aldo Matteucci 
Lecturer 

CDepa-y+^ent of Agricult\iral and Resource 

^ mics 
University of Ma. .and 
College Park, MD. 
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MASSACHUSETTS 
John H« Npyes 

Coommity Resource Program 

Coordinator 
Department of Forestry and Wildlife 

Management 
University of ^Massachusetts 
Amherts, MA 

Thomas F. Quink 

Rural Development Extension 

Specialist 
Department of Forestry and Wildlife 

Management 
University of Massachusetts 
Amherts, MA 

HEW HAMPSHIRE 

Edmund F» Jansen, Jr# 
Associate Professor of Resource 

Economics 
University of New Hampshire 
Durham, NH 

Silas Weeks 
Extension Economist i 
Institute of HaturaL and Environ- 
mental Resources 
University of New Hampshire 
Durham, HH 

NEW JERSEY 

* 

Merle Adams, Chairman 
Department of Agricultural and 

Resource Management Specialists 
Cooperative Extension Service 
Rutger^ University 
New Brunswick, NJ 

Henry Byers 

Department of Environmental 

Resources 
Rutgers University 
New Brunswick, NJ 
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NEW JERSEY (Continued) 

George H. Niesvand 

Associate Professor of Environmen- 
tal Systems Analysis 

Department of Environmental 
Resources 

Rutgers University 

Mew Brunswick, KJ 

KEH YORK 

Joan Egner, Associate Director 
Cornell University Agric\iltural 

Experiment Station 
Ithaca, NY 

Jane S* Gore 

Clinton County Rural Development 

Program Coordinator 
Miner Center 
Chazy, HY 

Helen Y. Nelson 

Professor of Canmmity Service 

Education 
Department of Conanunity Service 

Education 
Cornell University 
Ithaca, NT 

Carolyn Russell. 
County Coordinator 

^Jlinton County Cooperative Extension 
Piatt sburgh, HY 

Noland L. -VanDemark, Director 
Cornell University Agricultural 
Experiment Station 
Ithaca, NY 

Henry Hadsworth, Title V Coordinator 
Associate Director, Cooperative 

Extension 
Cornell University 
Ithaca, NY 

PENNSYLVANIA 

J. Patrick Madden 

Professor of Agrictiltural Economics 
Departmeot of Agricultural Economics 

and Rural Sociology 
Pennsylvania State University 
University Park, PA 



PENNSYLVANIA (Con't.) 

Kenneth M. Martin 
Cotmnunity Development Specialist 
Pennsylvania State University i 
University Park, PA 

Joan S. Thomson 
Title V Project Leader 
Pennsylvania State University 
University Park, PA 

Kenneth P. Wilkinson 
Associate Professor of Rural Sociology 
Pennsylvania State University 
University Park, PA ^ 

RHODE ISLAND 

Earl Patric, Associate Director 
Agricultural Experiment Station 
University of Rhode Island 
Kingston, RI 

Eliot C, Roberts, Chairman 
Department of Plant and Soil Science 
Rural Development Research and ^ 
Extension Project CoordjJiator 
University of Rhode Island 
Kingston, RI 
* 

VERMONT 

Ronald Bouf fard' 
Government Liaison Specialist 
Cooperative Extension Service 
Montpelier, VT 

WEST VIRGINIA 

Robert W. Miller 

Program Leader - Social and 

Economic Development 
Appalachian Center 
' West Virginia University 
Morgantown, WV 

Ernest J. Hesiu5 

Professor of Agricidtural Economics 
Division of Resource Manag'ement 
Title V Research Coordinator 
West Virginia University 
Morganxown, WV 
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U .S • DEPARTMENT OF AGRICULTURE 
Edward 0, Moe 

Principal Sociologist and Coordinator, 

Rural Development Programs 
Cooperative State Research Service 
U.S. Department of Agriculture 
Washington, DC 

Howard C. Tankersley 
Program Leader - Rural Pevelopajent 
Extension Title V Coordinator 
" Extension Service 
U.S. Department of Agriculture 
Washington, DC 

Emily P. Wood 
'"Extension Service 
U.S. Department of Agriculture 
Washington, DC ^ . 

WORTH CEMTRAL REGIONAL CENTER FOR RURAL DEVELOIMENT 

/ 

David L.' Rogers 

Iowa State University of Science and 

Technology 
Ames, Iowa 

NORTHEAST REGIONAL CENTER FOR/iuRAL DEVELOIMENT 
Lee Day, Director Designa-J;e 
Leslie C. Hyde, Assisteuit to the Director 
Olaf-F. Larson, Director 
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